Welcome to the project page of my GSoC 2017 project!
XMPP stands for
eXtensible Message and Presence Protocol. The focus lays on extensible.
Many features are not part of the core specification, but instead are
defined in so called XEPs (XMPP Extension Protocols). XEPs are numbered,
so every XEP has a unique number and a name to identify it. The goal of
my project was to implement XEP-0234 – Jingle File Transfer. In order
to achieve this goal, I had to implement a whole number of other XEPs,
since there are some transitive dependencies (see below).
I documented my progress in weekly blog posts. Those can be found in my blog.
XEP-0234 – Jingle File Transfer
This XEP describes how two entities can exchange files using the Jingle Protocol. Jingle itself is another XEP which I also had to implement, since Jingle File Transfer is defined on top of it. The specification for XEP-0234 can be found here. The code I wrote for that XEP is available in this package.
XEP-0166 – Jingle
The Jingle XEP defines how two entities can negotiate bytestreams between each other. This involves, which transport methods are used (examples for that down below), which parameters are used for the application of the bytestream (eg. which audio codec in a voice chat) and so on. Also security plays a role in the negotiation. The Jingle protocol consists of three main components:
- Description: Describes the application of the session (eg. file transfer – see XEP-0234)
- Transport: The transport method used. This might be InBandBytestreams, SOCKS5 proxies…
- Security: The encryption used. Describes how the bytestream is secured.
A session consists of one or more contents, which in turn consist of a description, a transport method and a security component. Those components can be composed and replaced in arbitrary ways.
Implementing the Jingle XEP was the biggest part of my project. I worked using the waterfall model to learn from previous errors I made and it took me 3 iterations, until I was happy with my code. Now I have a very modular codeset which can be easily extended and composed in any way. This blog post gives an overview of the structure. The specification is available here, while my implementation can be found here.
XEP-0261 – Jingle InBandBytestream Transport
The easiest, most basic transport method is described in XEP-261. When data is sent using InBandBytestreams, the raw data is split into blocks of typically 4096 base64 encoded byte, which are sent one after another via the XMPP protocol itself. This method is occasionally very slow, but since it depends only on the XMPP protocol itself, it does not require additional server infrastructure. The XEP can be found here, my implementation resides in this package.
XEP-0260 – Jingle SOCKS5Bytestream Transport
A more advanced transport method is described in Jingle SOCKS5Bytestream Transports (short Jingle S5B). Here SOCKS5 proxy servers are utilized to enabled clients behind firewalls or NATs to establish connections with each other. It is possible to use either external servers (eg. proxy components of the XMPP servers), or local servers for this purpose.
Working on the implementation for that XEP was a pain. For half a week I was stuck at finding one little bug. My frustrated blog post has more information about that. The XEP can be found here, my implementation here.
XEP-0300 – Use of Cryptographic Hash Functions
A Jingle File Transfer involves checksums to ensure the integrity of the sent file. Since computer science is a rapidly changing field, hash functions might quickly get obsolete (see SHA1 – well done you, Google :D), so it is desirable to be able to quickly replace hash functions. For that purpose XEP-0300 defines namespaces for hash functions, so that negotiators can use different algorithms as they wish. The specification can be found here, while my implementation lives here.
XEP-xxxx – Jingle Encrypted Transports (JET)
The most exciting part of my project was to create my own specification! With JET I specified a way to utilize OMEMO and OX (OpenPGP) end-to-end encryption to secure Jingle transports. This way entities can transfer bytestreams end-to-end encrypted. The specification allows for easy extensibility with other encryption mechanisms additionally to OMEMO or OX. My plan is to propose the specification to the XSF for approval, so that it might one day become an official XEP.
This blog post goes into more details on how the protocol works.
The following of my code has been merged into Smacks master branch. Some parts are still missing, which is partially due to experimental features, which need approval by the XSF. Unmerged code can be found in my repository.
- XEP-0166 – Jingle (first iteration)
- XEP-0260 – Jingle S5B (first iteration)
- XEP-0261 – Jingle IBB (first iteration)
- XEP-0300 – Hashes
Immediate Merge Pending
- XEP-0234 – Jingle File Transfer
- XEP-xxxx – Jingle Encrypted Transports (will be merged after XEP-approval by the XSF)
The whole point of XMPP is to avoid walled gardens, so there are many clients implementing many of the specified features. Ideally all clients are interoperable to one another (in reality it doesn’t always go so well unfortunately). This is why I’m very happy to announce that my file transfer code has been successfully tested against Gajim, a well established desktop XMPP client, as well as against Conversations, a popular XMPP client for Android.
Since my works is mostly invisible library code, I wrote a small application to demonstrate some features. XMPP_SYNC is a command line application, which can be used to synchronize files between computers via (encrypted) Jingle File Transfer.
I blogged about my work:
- My GSoC introduction
- GSoC: First week of community bonding
- GSoC: Second, third week of community bonding
- Last week of GSoC Community Bonding
- GSoC – Second week of coding
- Third Week of GSoC
- Fourth week of GSoC and OMEMO thoughts
- GSoC Week 4.1
- GSoC Week 5: Tests, fallbacks and politics
- GSoC Week 6 – Tests and Excitement
- GSoC: Week 7
- GSoC Week 8: Reworking
- GSoC Week 9: Bringing it back to life.
- GSoC Week 10: Finding that damn little bug
- GSoC Week 11: Practical Use
- GSoC Week 11.5: Success!
- Final GSoC Blog Post – Results
Feedback on XEPs
During my time implementing the XEPs, I often gave feedback to the XSFs standards mailing list.
- Encrypted Jingle File Transfers
- Ciphers XEP?
- XEP-0166 – Missing unsuported-security action
- XEP-0166 – Multiple Contents in transport-info message
- XEP-0234 – NonNegativeInteger
- Jingle XEPs – Use of session-info vs. description-info
- XEP-0234 – Wrong date format
- XEP-0260 – Redundant DstAddr
- XEP-0234 – Add option to request hash of offered file
- XEP-0234 – Denote used hash function in session-initiate
- XEP-0234 – Missing error condition for checksum mismatch
Whenever I could, I created Pull Request to address errors I found in the XEPs.
- XEP-0065 – Remove duplicate example
- Proto-XEP JET
- XEP-0234 – Fix xml in examples
- XEP-0234 – Add missing length attribute to XML schemas
- XEP-0234 – Fix wrong date format to conform with XEP-0082
- XEP-0234, XEP-0300 – Add hash-used element
Big thanks to all, who welcomed me 🙂