When I implemented Raft using Chirp, it turned out to be so great, I just love it. I also learned that what I am doing is actor-based programming, so I read about the actor-model and again I loved what I read. I checked how fast Erlang is for remote-actors and compared it with many existing solutions in Python: usually Erlang can send 1000x more messages per second. Then I said: "Solomon, if we are going to to this, lets go big!" I want Chrip to send at least 10x more msg/s, I hope for 100x msg/s.
So I implement Chirp in C using libuv. I call it C4irp. At least in a pure C agent, I'd like to see 1000x msg/s. Which means 1'000'000 msg/s on a single core.
Concrete Clouds C Chirp
- Chirp for any language and any Python version, specially PyPy
- CFFI python binding
- Two Python bindings, high-level: for speed, low-level for testing
- libuv
- mbedtls
It's symmetric, dude, symmetric
I try to be rational.
- Note
- I will reference ZeroMQ a lot, but I never used ZeroMQ in a project that went into production, because it didn't solve the problems I had to solve. So I might be misled.
I asked myself these questions:
- What does Chirp do?
- Is Chirp an unique and new approach and therefore worthwhile implementing it?
- Why do I like Chirp?
- What is the key property of Chirp?
If you answer the first question, the rest is quite easy to answer: Chirp makes the asymmetric TCP symmetric. The peer that contacted the other, is the TCP-Client and the other the TCP-Serer, but Chirp will use that stream to send messages back. There is no bind or connect, you just send messages. Of course Chirp hides bind and connect from the user.
So why is being symmetric a good thing?
- Most of the current messaging solution aren't symmetric
- Implementing asymmetric algorithms with symmetric messaging is easy
- Implementing symmetric algorithms with asymmetric messaging is a pain
- Implementing Raft with Chirp is a breeze
- Implementing Raft with ZeroMQ is cumbersome
- The patterns in ZeroMQ and Nanomsg are great, but they are imposed on the user. If you need something else, you have to fight against the library.
- In Concrete Clouds many of the ZeroMQ patterns will be implemented, but you are free to use them or just do simple symmetric messaging
- It just like UTP without most of the restrictions and TLS support
Concrete Clouds is a stack
For my taste all the libraries out there do too much. In Concrete Clouds Chirp has only one job: make TCP symmetric. Everything else is solved in other layers/modules. In my opinion every module should only do a single thing and be as small as possible. To solve complex problems you stack multiple modules.
But wait
You take the asymmetric TCP make it symmetric and then implement asymmetric messaging with it, sounds like a bad idea.
First making TCP symmetric comes almost for free, you just have to reuse the stream to send messages back. The only cost we have: Chirp keeps the TCP connection around until it was idle for N seconds. Usually connections are garbage-collected after 30s idling. The resources of the TCP connection are bound a bit longer than needed. In comparison in ZeroMQ you often need many TCP connections since you need multiple sockets for most real world solutions. So we actually use less resources than ZeroMQ.
Second making TCP symmetric comes with all sorts of nice free goodies:
- TLS support
- Message flow control
- Reliability: Messages cannot be lost without an exception
- Robustness
- Actor-model
Final word
Chirp probably isn't better than other solutions, but it is a different approach, which I think should be available.