You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: collections/_posts/2022-09-17-typelevel-native.md
+90-12Lines changed: 90 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,13 +38,13 @@ Christopher Davenport has put up a [scala-native-ember-example](https://github.c
38
38
39
39
The burden of cross-building the Typelevel ecosystem for Scala Native fell almost entirely to [Cats Effect] and [FS2].
40
40
41
-
#### event loops
41
+
#### event loop runtime
42
42
43
43
**To cross-build Cats Effect for Native we had to get creative** because Scala Native currently does not support multithreading (although it will in the next major release). This is a similar situation to the JavaScript runtime, which is also fundamentally single-threaded. But an important difference is that JS runtimes are implemented with an [event loop] and offer callback-based APIs for scheduling timers and performing non-blocking I/O.
44
44
45
45
Meanwhile, Scala Native core does not implement an event loop nor offer such APIs. There is the [scala-native-loop] project, which wraps the [libuv] event loop runtime, but we did not want to bake such an opinionated dependency into Cats Effect core.
46
46
47
-
Fortunately Daniel Spiewak had the fantastic insight that the “dummy runtime” I initially used to cross-build Cats Effect for Native could be reformulated into a legitimate event loop implementation.
47
+
Fortunately Daniel Spiewak had the fantastic insight that the “dummy runtime” which I created to initially cross-build Cats Effect for Native could be reformulated into a legitimate event loop implementation by extending it with the capability to “poll” for I/O events: a `PollingExecutorScheduler`.
48
48
49
49
The [`PollingExecutorScheduler`] implements both [`ExecutionContext`] and [`Scheduler`] and maintains two queues:
50
50
- a queue of tasks (read: fibers) to execute
@@ -55,7 +55,9 @@ It also defines an abstract method:
55
55
defpoll(timeout: Duration):Boolean
56
56
```
57
57
58
-
The idea of this method is very similar to `Thread.sleep()` except that besides sleeping it may also “poll” for I/O events. To demonstrate the API contract, consider invoking `poll(3.seconds)`:
58
+
The idea of this method is very similar to `Thread.sleep()` except that besides sleeping it may also “poll” for I/O events. It turns out that APIs like this are ubiquitous in C libraries that perform I/O.
59
+
60
+
To demonstrate the API contract, consider invoking `poll(3.seconds)`:
59
61
60
62
*I have nothing to do for the next 3 seconds. So wake me up then, or earlier if there is an incoming I/O event that I should handle. But wake me up no later!*
61
63
@@ -72,27 +74,85 @@ Thus, with tasks, timers, and the capability to poll for I/O, we can express the
72
74
-**There is at least one outstanding timer**. Call `poll(durationToNextTimer)`, so it will sleep until the next I/O event arrives or the timeout expires, whichever comes first.
73
75
-**There are no tasks to do and no outstanding timers.** Call `poll(Duration.Infinite)`, so it will sleep until the next I/O event arrives.
74
76
75
-
In fact, this is a very basic implementation of the [I/O Integrated Runtime Concept]. The grander idea is that every `WorkerThread` in the `WorkStealingThreadPool` that underpins the Cats Effect JVM runtime can run an event loop exactly like the one described above, for exceptionally high-performance I/O.
77
+
This algorithm is not a Cats Effect original: the [libuv event loop] works in essentially the same way. It is however a first step towards to the much grander Cats Effect [I/O Integrated Runtime Concept]. The big idea is that every `WorkerThread` in the `WorkStealingThreadPool` that underpins the Cats Effect JVM runtime can run an event loop exactly like the one described above, for exceptionally high-performance I/O.
76
78
77
79
#### non-blocking I/O
78
80
79
-
**So, how do we implement `poll`?** The bad news is that the answer is OS-specific, which is a large reason why projects such as libuv exist. Furthermore, the entire purpose of polling is to support non-blocking I/O, which falls outside of the scope of Cats Effect. This brings us to FS2, and specifically the [`fs2-io`] module.
81
+
**So, how do we implement `poll`?** The bad news is that the answer is OS-specific, which is a large reason why projects such as libuv exist. Furthermore, the entire purpose of polling is to support non-blocking I/O, which falls outside of the scope of Cats Effect. This brings us to FS2, and specifically the [`fs2-io`] module where we want to implement non-blocking TCP [`Socket`]s.
82
+
83
+
One such polling API is [epoll], available only on Linux:
84
+
85
+
```c
86
+
#include<sys/epoll.h>
87
+
88
+
intepoll_create1(int flags);
89
+
90
+
int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event);
91
+
92
+
int epoll_wait(int epfd, struct epoll_event *events,
93
+
int maxevents, int timeout);
94
+
```
95
+
96
+
After creating an epoll instance (identified by a file descriptor) we can register sockets (also identified by file descriptors) with `epoll_ctl`. Typically we will register to be notified of the “read-ready” (`EPOLLIN`) and “write-ready” (`EPOLLOUT`) events on that socket. Finally, the actual polling is implemented with `epoll_wait`, which sleeps until the next I/O event is ready or the `timeout` expires.
97
+
98
+
As previously mentioned, these sorts of polling APIs are ubiquitous and not just for working directly with sockets. For example, [libcurl](https://curl.se/libcurl/) (the C library behind the well-known CLI) exposes a function for polling for I/O on all ongoing HTTP connections.
99
+
100
+
```c
101
+
#include <curl/curl.h>
102
+
103
+
CURLMcode curl_multi_poll(CURLM *multi_handle,
104
+
struct curl_waitfd extra_fds[],
105
+
unsigned int extra_nfds,
106
+
int timeout_ms,
107
+
int *numfds);
108
+
```
109
+
110
+
Indeed, this function underpins the `CurlExecutorScheduler` in [http4s-curl].
111
+
112
+
On macOS and BSDs the [kqueue] API plays an analogous role to epoll. We will not talk about Windows today :)
113
+
114
+
Long story short, I did not want the FS2 codebase to absorb all of this cross-OS complexity. So in collaboration with Lee Tibbert we repurposed my cheeky [epollcat] experiment into an actual library implementing JDK NIO APIs (specifically, [`AsynchronousSocketChannel`] and friends). Since these are the same APIs used by the JVM implementation of `fs2-io`, it actually enables this code to be completely shared.
115
+
116
+
[epollcat] implements an `EpollExecutorScheduler` for Linux and a `KqueueExecutorScheduler` for macOS. They additionally provide an API for monitoring a socket file descriptor for read-ready and write-ready events.
These are then used to implement the callback-based `read` and `write` methods of the JDK `AsynchronousSocketChannel`.
129
+
130
+
It is worth pointing out that the JVM actually implements `AsynchronousSocketChannel` with an event loop as well. The difference is that on the JVM, this event loop is used only for I/O and runs on a separate thread from the compute pool used for fibers and the scheduler thread used for timers. Meanwhile, epollcat is an example of an I/O integrated runtime where fibers, timers, and I/O are all managed on a single thread.
80
131
81
132
#### TLS
82
133
83
-
**The final important piece of the cross-build puzzle was a [TLS] implementation** for `TLSSocket` and related APIs in FS2. Although this task was daunting, ultimately it was straightforward to directly integrate with [s2n-tls], which has a well-designed and well-documented API. This is effectively the only non-Scala dependency required to use the Typelevel stack on Native.
134
+
**The last critical piece of the cross-build puzzle was a [TLS] implementation** for [`TLSSocket`] and related APIs in FS2. Although the prospect of this was daunting, in the end it was actually fairly straightforward to directly integrate with [s2n-tls], which exposes a well-designed and well-documented C API. This is effectively the only non-Scala dependency required to use the Typelevel stack on Native.
135
+
136
+
Finally, special thanks to Ondra Pelech and Lorenzo Gabriele for cross-building [scala-java-time] and [scala-java-locales] for Native and David Strawn for developing [idna4s]. These projects fill important gaps in the Scala Native re-implementation of the JDK that were essential to seamless cross-building.
137
+
138
+
And ... that is pretty much it. **From here, any library or application that is built using Cats Effect and FS2 cross-builds for Scala Native effectively for free.** Three spectacular examples of this are:
84
139
85
-
And that is pretty much it. **From here, any library or application that is built using Cats Effect and FS2 cross-builds for Scala Native effectively for free.** Three spectacular examples of this are:
86
140
*[http4s] Ember, a server+client duo with HTTP/2 support
87
141
*[Skunk], a Postgres client
88
142
*[rediculous], a Redis client
89
143
144
+
These libraries in turn unlock projects such as [feral], [Grackle], and [smithy4s].
145
+
90
146
### What’s next and how can I get involved?
91
147
92
148
Please try the Typelevel Native stack! And even better deploy it, and do so loudly!
93
149
94
150
Besides that, here is a brain-dump of project ideas and existing projects that would love contributors. I am happy to help folks get started on any of these, or ideas of your own!
95
151
152
+
* Creating example applications, templates, and tutorials.
153
+
- If you are lacking inspiration, try cross-building existing examples such as [fs2-chat], [kitteh-redis], [Jobby].
154
+
- Spread the word: [you-forgot-a-percentage-sign-or-a-colon].
155
+
96
156
* Cross-building existing libraries and developing new, Typelevel-stack ones:
97
157
- Go [feral] and implement a pure Scala [custom AWS Lambda runtime] that cross-builds for Native.
98
158
- A pure Scala [gRPC] implementation built on http4s would be fantastic, even for the JVM. Christopher Davenport has published a [proof-of-concept][grpc-playground].
@@ -102,7 +162,7 @@ Besides that, here is a brain-dump of project ideas and existing projects that w
102
162
* Integrations with native libraries:
103
163
- I kick-started [http4s-curl] and would love to see someone take the reigns!
104
164
- An [NGINX Unit] server backend for http4s promises exceptional performance. [snunit] pioneered this approach.
105
-
- Using [quiche] for HTTP/3 looks fun!
165
+
- Using [quiche] for HTTP/3 looks yummy!
106
166
- An idiomatic wrapper for [SQLite]. See also [davenverse/sqlite-sjs#1] which proposes cross-platform API backed by Doobie on the JVM.
107
167
108
168
* Developing I/O-integrated runtimes:
@@ -113,12 +173,13 @@ Besides that, here is a brain-dump of project ideas and existing projects that w
113
173
* Tooling. Anton Sviridov has spear-headed two major projects in this area:
114
174
-[sbt-vcpkg] is working hard to solve the native dependency problem.
115
175
-[sn-bindgen] generates Scala Native bindings to native libraries directly from `*.h` header files. I found it immensely useful while working on http4s-curl, epollcat, and the s2n-tls integration in FS2.
176
+
- Also: we are _badly_ in need of a pure Scala port of the [Java Microbenchmark Harness]. Not the whole thing obviously, but just enough to run the existing Cats Effect benchmarks for example.
0 commit comments