GLOG: Um registo partilhado para suportar a construção de aplicações distribuídas

A distributed shared log has become a core component in the backend of any non-trivial business application. As a result, the performance of a distributed log service impacts the performance of the entire information system. At the conceptual level, the advantage of a shared log is that it offers an unified and consistent (totally ordered) view of the complex flow of events that are exchanged across multiple concurrent clients and servers. As a result, a distributed shared log can be used to support complex services, such as state-machine replication. At the implementation level, providing this unified view to producers and consumers (log entry writers and readers respectively) can easily become a bottleneck. To circumvent these bottlenecks associated with totally ordering events is a significant research challenge. The goal of this project is to design, implement, and evaluate a large-scale, geo-distributed, totally-ordered shared log that offers low latency, both for event producers and for event consumers. Offering low latency for producers requires a design that optimizes log insertions by removing event ordering costs, such as contacting a centralized sequencer or executing a consensus protocol, from the critical path of the producer. Offering low latency for consumers requires a design that not only removes unnecessary blocking times to consumers (when a consumer is forced to wait for an event without knowing whether this event is relevant to the consumer) but that also supports the concurrent processing of events that can commute with each other. Our design takes these requirements into consideration. Our log, named Global LOG, or simply GLOG, will provide the following features: - GLOG will support geo-distributed operation. This means that, even if the resulting log offers a total order of all events in the system, client nodes at different locations will be able to insert and read log entries from their local region in order to minimize (user-perceived) latency. - GLOG will support subscribers with different consistency requirements. Not all applications require events to be totally ordered. For instance, many applications can operate under causal consistency. Such applications will be allowed to consume concurrent events in parallel, without being constrained by the total order imposed by GLOG. - GLOG will support clients that are only concerned with a subset of the information in the log. Clients that ingest information from the log will be able to subscribe only to the events that are relevant for them. Still, if two given events are covered by the subscription of different clients, those clients will observed the same total order for those events (which is the ordered defined by GLOG). Thus GLOG will combine features of classical logs with features of publish-subscribe systems, in a unique combination of flexibility and global consistency. The project will also implement a geo-replicated, sharded, multi-consistent transactional key-value store on top of GLOG, that we name GLOG-KV. GLOG-KV will support partial replication and transactions with different semantics. Among the semantics to be supported by GLOG-KV we will include RedBlue [RedBlue] consistency and Non-Monotonic Snapshot Isolation [NMSI].