Realtime (or not) Traffic Replication with Gor

As an operations engineer, we all have various tools that we rely on heavily. Those could be for load testing, monitoring, code analysis or deployments. One tool that I wanted to touch on is gor. Gor is an amazing tool which allows you to replay captured traffic in either real time or from capture files. This is incredibly useful when rewriting applications or even testing refactored changes. You can spot bottlenecks before you push changes to production.

gor

Realtime Replication

Grab the latest build here and deploy or copy it up to machine that you want to be the replayer. The replayer will be mainly CPU bound. Once on this machine, you can run it with the following:

This will setup the replayer to listen on port 28020. Any incoming traffic to this port will have its host header rewritten to match example.org and it will then be sent to staging.example.org limiting to 10 requests/second. You could also change the –output-http line to “http://staging.example.org|10%” to send 10% of incoming requests. Remove |10 or |10% entirely to just mirror 100% of traffic. For performance reasons, I would set GOMAXPROCS equal to the number of CPUs on this server.

Deploy or copy the latest build to a server that you want to replicate traffic from.

This will setup gor to capture traffic from port 80 and to send it to the replayer which we started running in the previous step.

Once these are both running (always ensure the replayer is running first) you will see requests start to hit your target.

Captured Replication

However, if you want to playback traffic faster than 100%, or for whatever reason don’t want to mirror live production traffic then you can create a capture file on the source machine.

Run this for however long you want. It will capture data from port 80 and dump it to requests.gor

Copy this file to our replayer and we can play this back.

Very similar to the first time we ran the replayer. However this will use our requests.gor file as a source, and replay it back at 200% speed. So if your production system saw 100/requests/second this should replay it back at 200/requests/second.

Note that there are a lot of other things you can do with gor and the readme has a lot of great documentation around its use. I have successfully used gor to replicate over 400k/requests/minute from one service to a staging service. In this case, I actually had to run multiple replayers.

 

Leave a Reply

Your email address will not be published. Required fields are marked *