How to use the protocol implementation outside of this repository?

[thomaseizinger]

I am planning on building an android app where I would like to (re)-use the syncthing protocol.

What would be the best way to re-use the native Java implementation in this repository?
Do you plan on releasing that to maven central?

I am aware of jitpack but I am not sure if it is a suitable way? It seems like the releases target the version of syncthing-lite and not the underlying protocol implementation. Maybe the protocol bit could be extracted into its own repository?

[l-jonas]

Releasing it to somewhere else is not really planned. The unclean solution would be to copy the needed subprojects to your project. Using it somehwere else would not be a very good idea because it uses experimental Kotlin features (and Kotlin coroutines which are a problem if you use java).

The protocol bit was moved here because it makes it easier to implement new features in the client. This implies that breaking changes are not avoided (even for improvements).

The question is about which part of the protocol you're speaking. Some parts are expected to keep the way they are so releasing them would be possible.

[thomaseizinger]

I am planning to also write the app in Kotlin so that part shouldn't be a problem.
In a nutshell, the app should synchronize files between several phones. From what I understand, this means I will need the block exchange protocol and the relay part (since none of the devices can open a port by themselves).

[l-jonas]

That sounds bad. The relay part (and the block exchange protocol part) are only for connecting to other devices and pulling files. Pushing files does not work. I started working on this topic at https://github.com/l-jonas/syncthing-lite/tree/file-uploading, but sending index updates to other devices is not implemented there.

[imsodin]

This probably means that the "library part" is not yet complete, but there's nothing preventing sending indexes from happening, right? Also I believe publishing was a topic when merging the syncthing-java repo, just wasn't done because there wasn't any other consumer than the syncthing-lite app.

I had a quick look at the different packages/libraries (no idea of the java lingo for the `net.synthing.java.*" (core, bep, ...) parts) to try and figure out what's the purpose/separation between those, and was a bit confused. From my memory when did tiny contributions by updating the protocol stuff bep more or less just contained the protocol specifications and core did all the index exchange/syncing/..., i.e. all "operations". Now I found a lot of functionality in bep too. What's the "high level" separation between bep and core?

Edit: I maybe should I mentioned why I commented in the first place: I'd really like it if this project gets more momentum. It's generally cool to have another implementation of BEP and for android, it would in my opinion be huge if we got a native option with external SD support and more going. Having two apps, a lite one as existing and what you envision (which sounds like "the full thing") would be great. It's definitely a lot of work, but by sharing the work done on the protocol/syncing side of things, both projects can profit.

[l-jonas]

core is what's used by multiple components, bep is the connection handling, syncthing-client is a wrapper around it for "end users"

[thomaseizinger]

It would be my first syncthing contribution but I'd open to help out with the implementation as long as there is some guidance around how things should be done.

Is somebody willing to write up some issues around this topic?
Are there any cleanup tasks that need to be done first?

[l-jonas]

It's possible to subscribe to index updates at https://github.com/syncthing/syncthing-lite/blob/master/syncthing-bep/src/main/kotlin/net/syncthing/java/bep/index/IndexHandler.kt#L60.

At https://github.com/syncthing/syncthing-lite/blob/master/syncthing-bep/src/main/kotlin/net/syncthing/java/bep/connectionactor/Instance.kt#L140, it's required to send the index updates from the database first and then all new updates which can be queried from the IndexHandler (linked above). The ConnectionActor gets an IndexHandler which allows access to the database https://github.com/syncthing/syncthing-lite/blob/master/syncthing-bep/src/main/kotlin/net/syncthing/java/bep/index/IndexHandler.kt#L40.

One difficulty with that is that index updates should not prevent using the connection (some users have got huge indexes which must be received and sent after connecting the first time).

Lastly, there is the not working block pusher at https://github.com/syncthing/syncthing-lite/blob/master/syncthing-bep/src/main/kotlin/net/syncthing/java/bep/BlockPusher.kt. Index updates should not be sent directly. Instead, they should be added to the local database (from which they should be sent to all connections using the index update notification channels). The files to upload should be saved as file on disk and not as inputstream. (And it should be possible to know which of them should be uploaded for which file even after an App restart)

I'm not sure for listening at relay servers, but because both sides of a bep connection a relative symetric, I don't expect it to be to difficult. Eventually it helps to break up the connectionhandler code.

Edit: I suggest using https://github.com/l-jonas/syncthing-lite/tree/file-uploading as base because it adds saving the fields which are required to save all index fields.