Skip to content
This repository was archived by the owner on Jun 25, 2026. It is now read-only.

WIP basic cluster communication #35

Open
psFried wants to merge 73 commits into
base: master
Choose a base branch
from
Open

WIP basic cluster communication #35

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
f06424d
fix how status readers/writers are created and passed around when ini…
psFried Nov 20, 2017
d63a2bd
initialize FloController on startup, though it doesn't actually do an…
psFried Nov 22, 2017
5ff88c0
move engine initialization into a separate module
psFried Nov 22, 2017
56d136b
only execute OSX builds with stable rust since they take longer
psFried Nov 22, 2017
57b11bf
set exit code to 1 when running server fails with error
psFried Nov 22, 2017
151bdc8
increase timeout on expiration test to account for really slow OSX ma…
psFried Nov 22, 2017
5373124
WIP on cluster communication
psFried Nov 26, 2017
0c906d7
more WIP on cluster communication
psFried Nov 26, 2017
13dd6ae
create separate submodules for each message in flo-protocol
psFried Nov 28, 2017
d291cc1
cleanup compiler warnings in protocol and add tests for serializing s…
psFried Nov 28, 2017
abce683
remove unused SetBatchSize protocol message
psFried Nov 29, 2017
caee363
protocol level implementation of append entries
psFried Dec 2, 2017
e4c4106
add request vote messages to protocol
psFried Dec 2, 2017
092a1a9
keep a reference to the socket address of the current primary server
psFried Dec 3, 2017
a35ac27
change connection handler to process either protocol or control messages
psFried Dec 3, 2017
7de19a3
fix produce operation in flo-client cli
psFried Dec 7, 2017
b41d765
factor out spawning of connection handler so it can be reused for cre…
psFried Dec 7, 2017
2ea6081
more shuffling things aroung in preparation for outgoing connections
psFried Dec 9, 2017
364e92d
WIP on partition controller and managing peer connections, mostly jus…
psFried Dec 14, 2017
03969e5
remove peer replication connection module until we are actually ready…
psFried Dec 14, 2017
0e7ca67
add new fields to peer announce message so members can tell who the p…
psFried Dec 14, 2017
06ab54d
sketch in a real initialization flow for cluster state
psFried Dec 18, 2017
3d7be05
factor out trait objects for some of the cluster-related state manage…
psFried Dec 20, 2017
8383bb9
Send a Tick operation to the system controller at a random interval b…
psFried Dec 20, 2017
7e3b273
remove debug logging from travis builds
psFried Dec 20, 2017
36732c4
work basic cluster state and management into initialization
psFried Dec 26, 2017
2d6223c
connections established on startup with known cluster members
psFried Dec 29, 2017
2f25413
peer connections are completing the handshake
psFried Dec 29, 2017
405cb85
change connectionHandler to shutdown on Drop instead or Sink::close
psFried Dec 29, 2017
de6e49d
handle requestVote in connectionHandler
psFried Dec 29, 2017
5397954
connectionHandler now also deals with VoteResponse messages
psFried Dec 30, 2017
21c1461
controller triggers an election after the timeout, but of course ther…
psFried Dec 30, 2017
f0d89f3
factor out a PeerConnectionManager trait to make testing easier, impl…
psFried Dec 31, 2017
da6c05e
add doc comments to PartitionImpl fields so I remember wtf I was thin…
psFried Dec 31, 2017
89e2a56
minor test refactor, factor out function for determining the minimum …
psFried Dec 31, 2017
bbbd66d
start handling vote responses in flo controller
psFried Jan 1, 2018
3a8331e
ignore vote responses from unknown peers, restart election on timeout
psFried Jan 2, 2018
3619ff1
connection handler sends out append entries
psFried Jan 2, 2018
cf6b8ac
test sending heartbeat from connection handler, add commit index to C…
psFried Jan 2, 2018
2d0328d
handle receiving append entries including events
psFried Jan 3, 2018
81675a4
handle sending and receiving appendEntries responses in connection ha…
psFried Jan 4, 2018
1b3950a
rename FloEvent::to_owned to to_owned_event to differentiate it from …
psFried Jan 4, 2018
a0dda49
add basic skeleton of SystemEvent
psFried Jan 5, 2018
a7972cd
change how AppendEntries is sent so that peer offset is maintained by…
psFried Jan 7, 2018
1aa8d21
WIP on sending AppendEntries
psFried Jan 10, 2018
1e04eac
controller receives appendEntries and sends response, still not actua…
psFried Jan 20, 2018
44cfd28
primary is elected, seemingly correctly, but still no way to actually…
psFried Jan 21, 2018
b2610ec
tweak travis config to try to speed up builds
psFried Jan 21, 2018
6c73000
basic wiring in of commit manager into partitionImpl
psFried Jan 21, 2018
1c1b121
remove unused method from controller state, tell consensus processor …
psFried Jan 21, 2018
1041e56
start adding ability for partition to notify producers once events ar…
psFried Jan 21, 2018
5f0c8c6
prevent system stream from being re-initialized as a user stream, add…
psFried Jan 21, 2018
3bdf125
change default stream name to 'user' since the system stream is now l…
psFried Jan 21, 2018
80c2e75
backup commit, wip on replication operations
psFried Mar 12, 2018
0dc9b02
add crc function to flo-event
psFried Apr 29, 2018
8aaa098
get flo-server compiling again
psFried Apr 29, 2018
56a9bab
add a CRC to events
psFried May 6, 2018
1a49cc4
POC implementation of event replication is sort of working in partiti…
psFried May 13, 2018
5424d26
finish the firs basic replication test, make partitionReader explicit…
psFried May 13, 2018
8148956
server handles committed and uncommitted consumers separately and pro…
psFried May 13, 2018
aef29dc
add a commitIndexUpdated protocol message
psFried May 19, 2018
5f68b1d
change SystemEvent representation to include deserialized data
psFried May 20, 2018
824474d
back out system event additions, change FloInstanceId to be a typedef…
psFried Jun 9, 2018
c21e0e5
cleanup compiler warnings
psFried Jun 9, 2018
c09e15f
prevent users from producing invalid system events
psFried Jun 9, 2018
dbe87d0
add script to start cluster with multiple members
psFried Jun 10, 2018
51b3c58
minor improvements to dev-start-cluster.sh
psFried Jun 10, 2018
dd0c681
- Make all partitions start as non-writable when in cluster mode (sti…
psFried Jun 12, 2018
b8264e1
adjust system event types in prep for cluster membership rewrite
psFried Jun 13, 2018
84ca31b
start having system events applied to persistent state
psFried Jun 13, 2018
f910402
apply the remaining system event to persistent state
psFried Jun 14, 2018
ed17aa2
process acknowledgements from peers
psFried Jun 16, 2018
f7c94f0
backup commit - squash me
psFried Jun 24, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
sudo: false
language: rust
cache: cargo
env:
global:
- RUST_BACKTRACE=1
- RUST_LOG=flo=debug
rust:
- stable
- beta
Expand All @@ -13,4 +14,9 @@ os:
matrix:
allow_failures:
- rust: nightly
script: cargo test --all --no-fail-fast
exclude:
- rust: beta
os: osx
- rust: nightly
os: osx
script: cargo test --all
Loading