This is a pretty subjective post. I'm sharing my perspective, taking into account years of experience building backend and frontend with user experience in mind.
If you do not want to read this article, then watch the recording of the live stream about the same:
Everything we hear is an opinion, not a fact. Everything we see is a perspective, not the truth. ― Marcus Aurelius
This blog post is the first of a series of blog posts about WebSocket I'm working on.
What is WebSocket
It is a pretty old protocol used for duplex communication over TCP connection. It was standardized in 2011. Yes, ten years ago means it is old, super old.
So why do I even mention it in 2021?
It is very widely adopted and will not go away anytime soon because tooling support is excellent and serves its purpose well. Just remind yourself when HTTP/2 showed up and how many years it took everyone to migrate. It would not happen without the strong support and push from all the big players.
Sure, there is HTTP/2 multiplexing and protocols like Mercure or GraphQL Subscription. There is also RFC8441 for WebSocket and HTTP/2 and some tools already adopted it, like Envoy or Jetty. Nevertheless, WebSocket is here to stay.
Anyway, the future of WebSocket has nothing to do with this post. This post is for the AsyncAPI community looking into the AsyncAPI spec because of WebSockets now, no matter the protocol's future.
Websocket use case
- Do you like to see in Slack that someone is typing a response?
- Do you like it when a user interface updates without page refresh?
- Do you like it when your client app knows there are updates available for display?
That is what WebSocket is for. You establish a long-living connection between client and server. Through such a connection, the client can send a stream of messages to the server, and this is possible the other way around at the same time.
One could say: I don't need WebSocket to achieve that. I could just set up a data polling with REST API. Just ask the API every few seconds if there are updates.
Sadly this is not a joke. Engineers do it. Some engineers just take shortcuts, mostly because deadlines hunt them down.
HTTP polling was presented very well in Shrek's famous Are we there yet? scene.
Don't go that path. Do not perform unnecessary connections to your servers and create more and more traffic with more and more resource consumption. Wasting resources is bad and makes Shrek angry. WebSocket changes a lot there:
Figure 1: HTTP Pull vs WebSocket vs Shrek.
Why AsyncAPI
When building a WebSocket API on a server, you might have some additional needs:
- Want to document the API for the team that writes a client app, Web UI, Desktop app, or Mobile app.
- Want to have a way to specify the format of the messages that the server supports to validate them in the runtime.
- Want to generate a server or/and a client? If not for final production use, then for sure for prototyping and testing.
These are just a few common needs. For WebSocket, you only establish a connection over HTTP protocol, and the rest goes over WS, so OpenAPI specification won't help you much here. WebSocket is one of the patterns in event-based systems. In the end, it is all about a stream of messages and asynchronous processing. Yes, it would be best to use AsyncAPI 😃
WebSocket described with AsyncAPI
When I google for some public WebSocket API to play with, I find mostly currency trading products:
Currency trading is a topic I know nothing about 🤷♂ but it feels interesting to explore more. Documentation of the 1st and 2nd API looks familiar from look&feel perspective. I think we can make a bet they are already using AsyncAPI, and Kraken most probably is still running on version 1. Let's release the Kraken then.
I'm sorry if you expected me to describe Shrek's API interface using AsyncAPI. It would be fun, but only fun, and I'd also like to teach you something.
I will write an AsyncAPI document for Kraken API after playing with the API and basing it on the current documentation.
Playing with WebSocket API
The best way to play with a WebSocket API is through a CLI. Who didn't hear about curl in the REST API world? For WebSocket, I would recommend websocat. Kraken's API is partially public without authorization which is just great because to play with it, you do not have to set up an account to get an authorization token.
- Install websocat. For other installation options, check out this list.
brew install websocat- Establish connection with the API:
websocat wss://ws.kraken.com- Ping the API to see if it responds. Just type the below message and hit Enter:
{"event": "ping"}- Now subscribe to the event ticker stream that sends messages with currency price. Just type the below message and hit Enter:
{ "event": "subscribe", "pair": [ "XBT/USD", "XBT/EUR" ], "subscription": { "name": "ticker" }}- You should now see a constant stream of data sent by the server. You do not have to ask the API every second for an update, as the update is pushed to you.
1{"event":"heartbeat"}
2[340,{"a":["45520.10000",6,"6.78103490"],"b":["45520.00000",0,"0.00185230"],"c":["45520.10000","0.01643250"],"v":["1397.95434819","5589.12101024"],"p":["44883.49461","44062.07654"],"t":[14350,66782],"l":["43607.60000","42770.80000"],"h":["45811.10000","45811.10000"],"o":["43659.30000","44709.10000"]},"ticker","XBT/EUR"]
3[340,{"a":["45520.10000",5,"5.84803490"],"b":["45492.50000",0,"0.09374582"],"c":["45492.50000","0.00625418"],"v":["1398.10526819","5589.26685876"],"p":["44883.56109","44062.11477"],"t":[14359,66790],"l":["43607.60000","42770.80000"],"h":["45811.10000","45811.10000"],"o":["43659.30000","44709.10000"]},"ticker","XBT/EUR"]
4{"event":"heartbeat"}
5[340,{"a":["45503.80000",1,"1.00000000"],"b":["45496.20000",0,"0.01426600"],"c":["45496.20000","0.00109400"],"v":["1398.10636219","5589.26295766"],"p":["44883.56157","44062.11447"],"t":[14360,66788],"l":["43607.60000","42770.80000"],"h":["45811.10000","45811.10000"],"o":["43659.30000","44709.90000"]},"ticker","XBT/EUR"]
6{"event":"heartbeat"}Boy, it is always such fun to do it. Like seriously, I always have fun playing with APIs, any APIs. Just making this API "conversation". I hope nothing is wrong with me 😅
Now you know how to interact with the Kraken API. Now let's try to describe it using AsyncAPI.
Describing API using AsyncAPI
I'll explain, in detail, how to describe Websocket API with AsyncAPI in another blog post that will be part of the series. Why? I don't want to make this post super lengthy and discourage others from reading it. Let us learn step by step.
For now, I will throw here a full AsyncAPI document I created for the Kraken API. You can also open it up in the AsyncAPI Studio and compare with their current documentation
Familiarize with below before you look at the AsyncAPI document:
- AsyncAPI describes the API interface between the client and the server. In other words, the AsyncAPI document is for the user of the API. It does not describe what the server does but what the user can do with the API.
- Kraken API is quite complex. It has some beta servers, some private messages, and messages closely related to vocabulary specific for currency trading. I dropped all of those from my research not to overcomplicate things. In other words, the AsyncAPI file that you can see below is not a complete document.
- Websocket protocol is very flexible, and therefore you can implement the server in many different ways. There is no standard way of doing things, like there is no common way of doing things with AsyncAPI. We can only make some generic assumptions looking at existing implementations:
- Your server has one entry point, just one endpoint that you communicate with to gain access to the API. It can be a path with some dynamic values, as some data id. It can also be nothing, no path at all, like in the case of below Kraken API. These entry points are channels in AsyncAPI document. Commonly, Websocket API has just one channel that user can send messages to and receive messages at the same time
- AsyncAPI publish and subscribe operations translates to messages user can send to the API and messages user will receive from the API. Depending on API complexity, sometimes you have an API that sends only one message. You can also have a situation where you can send to the server multiple different messages, and also receive different messages in response. This is when you need to use oneOf as I did in document for Kraken API.
- Current AsyncAPI limitation is that you cannot specify that once the user sends (publish) message ping, the pong message is a reply. Look at this thread to participate in an ongoing discussion about request/reply pattern support in AsyncAPI. In the below document, you will notice that for such a use case, I use AsyncAPI specification extensions (x-response).
Message to Kraken API developers and technical writers
In case you want to continue the work I started on the AsyncAPI document for Kraken API, feel free to do that. I'm happy to help, just let me know. Reach me out in our AsyncAPI Slack workspace.
1asyncapi: 2.0.0
2
3info:
4 title: Kraken Websockets API
5 version: '1.8.0'
6 description: |
7 WebSockets API offers real-time market data updates. WebSockets is a bidirectional protocol offering fastest real-time data, helping you build real-time applications. The public message types presented below do not require authentication. Private-data messages can be subscribed on a separate authenticated endpoint.
8
9 ### General Considerations
10
11 - TLS with SNI (Server Name Indication) is required in order to establish a Kraken WebSockets API connection. See Cloudflare's [What is SNI?](https://www.cloudflare.com/learning/ssl/what-is-sni/) guide for more details.
12 - All messages sent and received via WebSockets are encoded in JSON format
13 - All decimal fields (including timestamps) are quoted to preserve precision.
14 - Timestamps should not be considered unique and not be considered as aliases for transaction IDs. Also, the granularity of timestamps is not representative of transaction rates.
15 - At least one private message should be subscribed to keep the authenticated client connection open.
16 - Please use REST API endpoint [AssetPairs](https://www.kraken.com/features/api#get-tradable-pairs) to fetch the list of pairs which can be subscribed via WebSockets API. For example, field 'wsname' gives the supported pairs name which can be used to subscribe.
17 - Cloudflare imposes a connection/re-connection rate limit (per IP address) of approximately 150 attempts per rolling 10 minutes. If this is exceeded, the IP is banned for 10 minutes.
18 - Recommended reconnection behaviour is to (1) attempt reconnection instantly up to a handful of times if the websocket is dropped randomly during normal operation but (2) after maintenance or extended downtime, attempt to reconnect no more quickly than once every 5 seconds. There is no advantage to reconnecting more rapidly after maintenance during cancel_only mode.
19
20servers:
21 public:
22 url: ws.kraken.com
23 protocol: wss
24 description: |
25 Public server available without authorization.
26 Once the socket is open you can subscribe to a public channel by sending a subscribe request message.
27 private:
28 url: ws-auth.kraken.com
29 protocol: wss
30 description: |
31 Private server that requires authorization.
32 Once the socket is open you can subscribe to private-data channels by sending an authenticated subscribe request message.
33
34 The API client must request an authentication "token" via the following REST API endpoint "GetWebSocketsToken" to connect to WebSockets Private endpoints. For more details read https://support.kraken.com/hc/en-us/articles/360034437672-How-to-retrieve-a-WebSocket-authentication-token-Example-code-in-Python-3
35
36 The resulting token must be provided in the "token" field of any new private WebSocket feed subscription: { "event": "subscribe", "subscription": { "name": "ownTrades", "token": "WW91ciBhdXRoZW50aWNhdGlvbiB0b2tlbiBnb2VzIGhlcmUu" } }
1
2channels:
3/:
4publish:
5description: Send messages to the API
6operationId: processReceivedMessage
7message:
8 oneOf:
9 - $ref: '#/components/messages/ping'
10 - $ref: '#/components/messages/subscribe'
11 - $ref: '#/components/messages/unsubscribe'
12
13subscribe:
14description: Messages that you receive from the API
15operationId: sendMessage
16message:
17 oneOf:
18 - $ref: '#/components/messages/pong'
19 - $ref: '#/components/messages/heartbeat'
20 - $ref: '#/components/messages/systemStatus'
21 - $ref: '#/components/messages/subscriptionStatus'
22
23components:
24messages:
25ping:
26summary: Ping server to determine whether connection is alive
27description: Client can ping server to determine whether connection is alive, server responds with pong. This is an application level ping as opposed to default ping in websockets standard which is server initiated
28payload:
29 $ref: '#/components/schemas/ping'
30x-response:
31 $ref: '#/components/messages/pong'
32heartbeat:
33description: Server heartbeat sent if no subscription traffic within 1 second (approximately)
34payload:
35 $ref: '#/components/schemas/heartbeat'
36pong:
37summary: Pong is a response to ping message
38description: Server pong response to a ping to determine whether connection is alive. This is an application level pong as opposed to default pong in websockets standard which is sent by client in response to a ping
39payload:
40 $ref: '#/components/schemas/pong'
41systemStatus:
42description: Status sent on connection or system status changes.
43payload:
44 $ref: '#/components/schemas/systemStatus'
45examples:
46 - payload:
47 connectionID: 8628615390848610000
48 event: systemStatus
49 status: online
50 version: 1.0.0
51subscribe:
52description: Subscribe to a topic on a single or multiple currency pairs.
53payload:
54 $ref: '#/components/schemas/subscribe'
55examples:
56 - payload:
57 event: subscribe
58 pair:
59 - XBT/USD
60 - XBT/EUR
61 subscription:
62 name: ticker
63 - payload:
64 event: subscribe
65 subscription:
66 name: ownTrades
67 token: WW91ciBhdXRoZW50aWNhdGlvbiB0b2tlbiBnb2VzIGhlcmUu
68x-response:
69 $ref: '#/components/messages/subscriptionStatus'
70unsubscribe:
71description: Unsubscribe, can specify a channelID or multiple currency pairs.
72payload:
73 $ref: '#/components/schemas/subscribe'
74examples:
75 - payload:
76 event: unsubscribe
77 pair:
78 - XBT/EUR
79 - XBT/USD
80 subscription:
81 name: ticker
82 - payload:
83 event: unsubscribe
84 subscription:
85 name: ownTrades
86 token: WW91ciBhdXRoZW50aWNhdGlvbiB0b2tlbiBnb2VzIGhlcmUu
87x-response:
88 $ref: '#/components/messages/subscriptionStatus'
89subscriptionStatus:
90description: Subscription status response to subscribe, unsubscribe or exchange initiated unsubscribe.
91payload:
92 $ref: '#/components/schemas/subscriptionStatus'
93examples:
94 - payload:
95 channelID: 10001
96 channelName: ohlc-5
97 event: subscriptionStatus
98 pair: XBT/EUR
99 reqid: 42
100 status: unsubscribed
101 subscription:
102 interval: 5
103 name: ohlc
104 - payload:
105 errorMessage: Subscription depth not supported
106 event: subscriptionStatus
107 pair: XBT/USD
108 status: error
109 subscription:
110 depth: 42
111 name: book
112
113schemas:
114ping:
115type: object
116properties:
117 event:
118 type: string
119 const: ping
120 reqid:
121 $ref: '#/components/schemas/reqid'
122required:
123 - event
124heartbeat:
125type: object
126properties:
127 event:
128 type: string
129 const: heartbeat
130pong:
131type: object
132properties:
133 event:
134 type: string
135 const: pong
136 reqid:
137 $ref: '#/components/schemas/reqid'
138systemStatus:
139type: object
140properties:
141 event:
142 type: string
143 const: systemStatus
144 connectionID:
145 type: integer
146 description: The ID of the connection
147 status:
148 $ref: '#/components/schemas/status'
149 version:
150 type: string
151status:
152type: string
153enum:
154 - online
155 - maintenance
156 - cancel_only
157 - limit_only
158 - post_only
159subscribe:
160type: object
161properties:
162 event:
163 type: string
164 const: subscribe
165 reqid:
166 $ref: '#/components/schemas/reqid'
167 pair:
168 $ref: '#/components/schemas/pair'
169 subscription:
170 type: object
171 properties:
172 depth:
173 $ref: '#/components/schemas/depth'
174 interval:
175 $ref: '#/components/schemas/interval'
176 name:
177 $ref: '#/components/schemas/name'
178 ratecounter:
179 $ref: '#/components/schemas/ratecounter'
180 snapshot:
181 $ref: '#/components/schemas/snapshot'
182 token:
183 $ref: '#/components/schemas/token'
184 required:
185 - name
186required:
187 - event
188unsubscribe:
189type: object
190properties:
191 event:
192 type: string
193 const: unsubscribe
194 reqid:
195 $ref: '#/components/schemas/reqid'
196 pair:
197 $ref: '#/components/schemas/pair'
198 subscription:
199 type: object
200 properties:
201 depth:
202 $ref: '#/components/schemas/depth'
203 interval:
204 $ref: '#/components/schemas/interval'
205 name:
206 $ref: '#/components/schemas/name'
207 token:
208 $ref: '#/components/schemas/token'
209 required:
210 - name
211required:
212 - event
213subscriptionStatus:
214type: object
215oneOf:
216 - $ref: '#/components/schemas/subscriptionStatusError'
217 - $ref: '#/components/schemas/subscriptionStatusSuccess'
218subscriptionStatusError:
219allOf:
220 - properties:
221 errorMessage:
222 type: string
223 required:
224 - errorMessage
225 - $ref: '#/components/schemas/subscriptionStatusCommon'
226subscriptionStatusSuccess:
227allOf:
228 - properties:
229 channelID:
230 type: integer
231 description: ChannelID on successful subscription, applicable to public messages only.
232 channelName:
233 type: string
234 description: Channel Name on successful subscription. For payloads 'ohlc' and 'book', respective interval or depth will be added as suffix.
235 required:
236 - channelID
237 - channelName
238 - $ref: '#/components/schemas/subscriptionStatusCommon'
239subscriptionStatusCommon:
240type: object
241required:
242 - event
243properties:
244 event:
245 type: string
246 const: subscriptionStatus
247 reqid:
248 $ref: '#/components/schemas/reqid'
249 pair:
250 $ref: '#/components/schemas/pair'
251 status:
252 $ref: '#/components/schemas/status'
253 subscription:
254 required:
255 - name
256 type: object
257 properties:
258 depth:
259 $ref: '#/components/schemas/depth'
260 interval:
261 $ref: '#/components/schemas/interval'
262 maxratecount:
263 $ref: '#/components/schemas/maxratecount'
264 name:
265 $ref: '#/components/schemas/name'
266 token:
267 $ref: '#/components/schemas/token'
268interval:
269type: integer
270description: Time interval associated with ohlc subscription in minutes.
271default: 1
272enum:
273 - 1
274 - 5
275 - 15
276 - 30
277 - 60
278 - 240
279 - 1440
280 - 10080
281 - 21600
282name:
283type: string
284description: The name of the channel you subscribe too.
285enum:
286 - book
287 - ohlc
288 - openOrders
289 - ownTrades
290 - spread
291 - ticker
292 - trade
293token:
294type: string
295description: base64-encoded authentication token for private-data endpoints.
296depth:
297type: integer
298default: 10
299enum:
300 - 10
301 - 25
302 - 100
303 - 500
304 - 1000
305description: Depth associated with book subscription in number of levels each side.
306maxratecount:
307type: integer
308description: Max rate-limit budget. Compare to the ratecounter field in the openOrders updates to check whether you are approaching the rate limit.
309ratecounter:
310type: boolean
311default: false
312description: Whether to send rate-limit counter in updates (supported only for openOrders subscriptions)
313snapshot:
314type: boolean
315default: true
316description: Whether to send historical feed data snapshot upon subscription (supported only for ownTrades subscriptions)
317reqid:
318type: integer
319description: client originated ID reflected in response message.
320pair:
321type: array
322description: Array of currency pairs.
323items:
324 type: string
325 description: Format of each pair is "A/B", where A and B are ISO 4217-A3 for standardized assets and popular unique symbol if not standardized.
326 pattern: '[A-Z\s]+\/[A-Z\s]+'Personal note
If you can, if you are in a planning phase, new project, etc., then start designing your architecture with AsyncAPI. Don't do the mistake of coding first and then trying to figure out how to describe it with AsyncAPI 😅
Stay tuned for the next blog post that guides you step by step through the above document ☮️
I recommend you also read another article from the series about WebSocket: Creating AsyncAPI for WebSocket API - Step by Step.