免费视频淫片aa毛片_日韩高清在线亚洲专区vr_日韩大片免费观看视频播放_亚洲欧美国产精品完整版

ZeroMQis a messaging library, which allows you to design a complexcommunication system without much effort. It has been wrestling with howto effectively describe itself in the recent years. In the beginning itwas introduced as ‘messaging middleware’ later they moved to ‘TCP onsteroids’ and right now it is a ‘new layer on the networking stack’.

Ihad some trouble understanding ZeroMQ at first and really had to resetmy brain. First of all, it is not a complete messaging system such as RabbitMQ or ActiveMQ. I know the guys of Linden Research compared them,but it is apples and oranges. A full fledged messaging system gives youan out of the box experience. Unwrap it, configure it, start it up andyou’re good to go once you have figured out all its complexities.

ZeroMQis not such a system at all; it is a simple messaging library to beused programmatically. It basically gives you a pimped socket interfaceallowing you to quickly build your own messaging system.

Float like a butterfly, sting like a bee

Butwhy use ZeroMQ and not just use the low level Berkeley socket interfaceor a high level messaging system? I think the answer is balance. Youprobably want the flexibility and performance of the low level whilestill having the ease of implementation of the high level. However,maintaining raw sockets is difficult and cumbersome when you want toimplement a scalable system. A high level system often works perfect ifyou use it for the situation it was designed for, but it can bedifficult to change core elements of the system and its ease of useoften comes with a cost in performance. This isn’t a problem that islimited to messaging systems only. We can see the previous dilemma alsoin web frameworks; it could very well be that this is exactly the reasonwhy ‘Micro Frameworks’ gain in popularity.

I believe that ZeroMQ perfectly fits this gap between the high and the low level, so what are its features?

Performance

ZeroMQis blazing fast. It is orders of magnitude faster than most AMQPmessaging systems and it can obtain this high performance because of thefollowing techniques:

• It does not have the overhead of an over-engineered protocol such as AMQP
• It can make use of efficient transports such as reliable Multicast or the Y-suite IPC transport
• It makes use of intelligent message batching. This allows 0MQ to efficiently utilize a TCP/IP connection by minimizing not only protocol overhead but also system calls.

Simplicity

TheAPI is deceptively simple, and it makes sending messages really simplecompared with a raw socket implementation where you have to continuously‘feed’ the socket buffer. In ZeroMQ you can just fire off an async sendcall, it will queue the message in a separate thread and do all thework for you. Because of this async nature, your application does nothave to waste time waiting until the message has been flushed. Theasync nature of 0MQ makes it a perfect companion for an event-basedframework.

ZeroMQ’s simple wire protocol fits perfectly in thecurrent time setting where we have lots of different transportprotocols. With AMQP it always felt a bit weird to use an extra protocollayer on top. 0MQ gives you complete freedom on how you encode yourmessage, as it will just interpret it as a blob. So you can send simple JSON messages, go the binary route with for example BSON, Protocol Buffers or Thrift and all this without feeling guilty.

Scalability

WhileZeroMQ sockets look low level they provide lots of features. A singleZeroMQ socket can for example connect to multiple end points andautomatically load balance messages over them. Or it can work as somesort of Fan-In, collecting messages from multiple sources through asingle socket.

ZeroMQ follows a brokerless designso that there is no single point of failure. Combine this with itssimplicity and performance and you get something that you can use tomake your application distributed.

Implementing a messaging layer with ZeroMQ

Inthe next section I will show how to design and implement a messaginglayer with ZeroMQ. For the code example I will use Brian Granger’s PyZMQ, which is the excellent Python binding to ZeroMQ.

Implementing a ZeroMQ messaging layer is a three-step approach:

1. Choose a transport
2. Set up the infrastructure
3. Select a messaging pattern

Choosing a transport

The first step is to choosing a transport. ZeroMQ provides 4 different transports:

1. INPROC an In-Process communication model
2. IPC an Inter-Process communication model
3. MULTICAST multicast via PGM, possibly encapsulated in UDP
4. TCP a network based transport

The TCPtransport is often the best choice, it is very performant and robust.However, when there is no need to cross the machine border it can beinteresting to look at the IPC or INPROC protocol to lower the latency even more. The MULTICASTtransport can be interesting in special cases. But personally, I am abit careful with applying multicast, as it is difficult to understandhow it will behave when scaling up. Think of issues such as figuring outhow many multicast groups you can create with this or that hardware andhow much stress it is going to put on the different switches in yournetwork. If you want to be sure that your code runs cross platforms itis probably best to go with TCP as the other transports are not guaranteed to be available on the different platforms.

Setting up the infrastructure

Whenyou have decided upon your transport you will have to think about howthe different components are connected to each other. It is simplyanswering the question: “Who connects to whom?” You probably want themost stable part of the network to BIND on a specific port and have the more dynamic parts CONNECT to that. In the image below we have depicted how a server binds to a certain port and how a client connects to it.

Itis possible that both ends of the networks are relatively dynamic sothat it is difficult to have a single stable connection point. If thisis the case, you could make use of the forwarding devices that ZeroMQprovides. These devices can bind to 2 different ports and forwardmessages from one end to the other. By doing so, the forwarding devicecan become the stable point in your network where each component canconnect to. ZeroMQ provides three kinds of devices:

1. QUEUE, a forwarder for the request/response messaging pattern
2. FORWARDER, a forwarder for the publish/subscribe messaging pattern
3. STREAMER, a forwarder for the pipelined messaging pattern

Inthe image below we can see such a device being used, in this situationboth the client and the server initialize a connection to the forwarder,which binds to two different ports. Using such a device will remove theneed of extra application logic, as you will not need to maintain alist of connected peers.

Selecting a message pattern

Theprevious steps build the infrastructure but did not specify the messageflow. The next step is to think carefully about the message patterneach component should follow. The patterns that 0MQ supports are:

1. REQUEST/REPLY, bidirectional, load balanced and state based
2. PUBLISH/SUBSCRIBE, publish to multiple recipients at once
3. UPSTREAM / DOWNSTREAM, distribute data to nodes arranged in a pipeline
4. PAIR, communication exclusively between peers

I will explain them a bit more below.

Request Reply

Therequest reply paradigm is very common and can be found in most type ofservers. For example: HTTP, POP or IMAP. This pattern has a certainstate associated with it as a request has to be followed by a reply. Theclient uses a socket of type REQ as it will initiate the request by performing a .send() on the socket. The server uses a socket of type REP, and it will start by performing a .recv() to read the incoming request, after which it can send its reply.

ZeroMQgreatly simplifies this pattern by allowing you to have a single socketconnect to multiple end points. ZeroMQ will automatically balancerequests over the different peers.

The Python code below will create an echo server that listens on port 5000 with a REP socket. It will then loop an alternation of performing .recv() for incoming requests and then .send() a reply to them.

Whenyou have multiple clients connected to this server the ZMQ socket willfair queue between all incoming requests. Now, if you want your clientto be able to connect to multiple servers as well, you can take theabove code, change port 5000 to 6000 and use it to run an extra server.The following client code will then be able to use both of the servers:

Theabove sends 10 requests in total but since we are connected to 2different servers, each server only has to handle 5 requests. Isn’t thatgreat? With only a few lines of code we were able to create adistributed client/server model.

Now, if we want to add an extraserver to handle our requests we will have to adjust our code. This canbe cumbersome as we need to do this for all our clients to let them knowit can now balance the requests over an extra server.

Thisis exactly where the ZeroMQ devices fit in. Instead of having theclients connect directly to multiple servers it can connect to a singleforwarding device. The forwarding device will then reroute all messagesto the connected servers.

Example client output:

Sending msg 0
Sending msg 1
Sending msg 2
Sending msg 3
Sending msg 4
Sending msg 5
Sending msg 6
Sending msg 7
Sending msg 8
Sending msg 9

Example output server 1 at port 5000:

Got msg 0
Got msg 2
Got msg 4
Got msg 6
Got msg 8

Example output server 2 at port 6000:

Got msg 1
Got msg 3
Got msg 5
Got msg 7
Got msg 9

Publish Subscribe

ThePub/Sub paradigm has gained lots of interest the last few years. Youcan think of things such as message pushing, XMPP or webhooks. In apub/sub pattern the components are loosely coupled. This will greatlyhelp you to scale out as there is no need to worry about thesubscribers. However, this loose coupling can also lead to unexpectedbehavior when not fully understood. A nice metaphor for the Pub/Subparadigm is thinking of it is a radio station. When you publish messagesyou send something over a certain frequency, only listeners that havesubscribed to that frequency will receive the signal. But also, just aswith a radio, if you tuned in to the station after the broadcast youwill miss the show.

It is good to stress that the various messagepatterns have no coupling with the infrastructure. It is thus possibleto bind to a port and publish to the peers that connect to it. But it isalso possible to do it the other way around, connect to multiple peersand broadcast to them. The first example resembles the radio metaphor(everybody can tune in), while the second one more resembles yelling atyour peers through a megaphone (a selected group). In both situationsyour peers can decide not to listen to your messages by not subscribingto them.

The following code shows how you could create a broadcasting server for live soccer events:

The server will generate an unlimited amount of events for the different countries and pushes them over a socket of type PUB. Below you can find some example output:

-> portugal corner
-> portugal yellow card
-> portugal goal
-> netherlands yellow card
-> germany yellow card
-> brazil yellow card
-> portugal goal
-> germany corner
…

Nowif we are only interested in events concerning The Netherlands andGermany we can create a client that subscribes to those specificmessages:

The client will create a SUBsocket, connect to our broadcast server at port 5000 and subscribe tomessages starting with ‘netherlands’ or ‘germany’. The output will looksomething like this:

netherlands red card
netherlands goal
netherlands red card
germany foul
netherlands yellow card
germany foul
netherlands goal
netherlands corner
germany foul
netherlands corner
…

Pipelining

Thepipeline pattern looks remarkably similar to the Rep/Req pattern, thedifference is that instead of requiring a reply being sent to therequester the reply can be pushed down the pipe. This is a paradigmcommonly seen when there is a need to process data in parallel. Forexample, lets say we have some sort of system that does facerecognition. We have a job server that pushes the images to one of theworkers, which will then process it, once finished it will then push itdown the stream again towards some sort of collector.

In the design at the left we can see that a worker will receive its message from an UPSTREAM socket and once they are processed sends them DOWNSTREAM. It routes messages from two different socket types.

The jobserver can just keep pushing tasks DOWNSTREAM througha single socket but with multiple endpoints. ZeroMQ and recently alsoPyZMQ can send the messages in a zero-copy manner. This is great if youneed to push large messages around and you don’t want to waste IOcycles.

Paired sockets

Pairedsockets are very similar to regular sockets as the communication isbidirectional, there is no specific state stored within the socket andthere can only be one connected peer. Most real life problems can becaptured in one of the previously explained patterns and I want torecommend that you look at them first before applying this one as itwill simplify your problem.

The figure at the left depicts theinfrastructure of a paired socket, the server listens on a certain portand a client connects to it. The red lines indicate the flow ofmessages, in this pattern both endpoints use a socket of type PAIR and as you can see the messages can flow bidirectional.

The following code shows how to implement such a thing. We will bind to a port on one end:

And on the other end where we will connect to it.

ZeroMQ and the future

Inthis post I have given a short introduction to ZeroMQ, I hope that atthis point you will now share my ideas about what a great little libraryit is. But while the library may feel small it has a grand vision ofbeing the new messaging layer. And really, it is not that weirdwhen you come to think of it. Scalability issues are mostly justcommunication and portability issues, ZeroMQ can solve these problemsfor you.

Lets say you want to create some new sort of databasebecause Redis, Cassandra, TokyoTyrant, Postgres, MongoDB, DabbleDB,CouchDB, HBase, etc. just don’t serve your needs that well. You createan amazing in memory tree representation for your data and have ablazing fast indexer. Now all you need is some sort of messaging layersuch that different clients can talk to your server. Preferablyimplemented in different programming language and with clusteringcapabilities. You could of course create such a messaging framework allby yourself, but that is a lot of hard work.

A simple solution isto just implement your database as a ZeroMQ server and pick a messageprotocol (fe JSON). As you have seen by now, implementing suchfunctionality with ZeroMQ is really easy and on top of this you will getalmost instant scalability because of the way ZeroMQ can routemessages. It will also make it incredibly easy to implement differentclients that will communicate with your server. Basically all you needto do is pick one of the 15 available language bindings, use the samemessage protocol and you’re done. Currently the following languages havea ZeroMQ binding: Ada, C, C++, Common Lisp, Erlang, Go, Haskell, Java,Lua, .NET, OOC, Perl, PHP, Python and Ruby.

ZeroMQcould very well be the new way in how we connect our components. A goodexample of someone who understands the possibilities of ZeroMQ is ZedShaw as can be seen with his recent project Mongrel2.You can use Mongrel2 to bridge the gap between a regular HTTP clientand a ZeroMQ component. If you don’t immediately see how awesome this isyou probably have never worked with websockets, comet or flash basedsockets. Another way to look at the great possibilities of such animplementation is to think of Facebook’s BigPipe where each Pagelet can transparantly be generated by a different component connected with 0MQ.