Temporal .NET Driver has reached feature complete
published on 2023/05/05
Different language runtimes have different trade-offs for writing workflows. Go is very fast and resource efficient due to runtime-supported coroutines, but that comes at the expense of type safety (even generics as implemented in Go are limited for this use). Java is also very fast and type safe, but a bit less resource efficient due to the lack of runtime-supported coroutines (but virtual threads are coming). It might sound weird to say, but our dynamic languages of JS/TypeScript and Python are probably the most type-safe SDKs when used properly; however, as can be expected, they are not the most resource efficient. .NET provides the best of all worlds: high performance like Go/Java, good resource utilization like Go, and high quality type-safe APIs.
We are estatic that Temporal finally provides first class support for .NET ecosystem especially since one of the co-founder of Temporal was instrumental in building Azure Service Bus and Azure Durable Task Framework.
For those not familiar with workflows as code, a workflow is a method that is executed in a way that can't fail—each step the program takes is persisted, so that if execution is interrupted (the process crashes or machine loses power), execution will be continued on a new machine, from the same step, with all local/instance variables, the call stack, and threads intact. It also transparently retries network requests that fail.
So it's great for any code that you want to ensure reliably runs, but having methods that can't fail also opens up new possibilities, like you can:
- Write a method that implements a subscription, charging a card and sleeping for 30 days in a loop. The await Workflow.DelayAsync(TimeSpan.FromDays(30)) is transparently translated into a persisted timer that will continue executing the method when it goes off, and in the meantime doesn't consume resources beyond the timer record in the database.
- Store data in variables instead of a database, because you can trust that the variables will be accurate for the duration of the method execution, and execution spans server restarts!
- Write methods that last indefinitely and model an entity, like a customer, that maintains their loyalty program points in an instance variable. (Workflows can receive RPCs called Signals and Queries for sending data to the method ("User just made a purchase for $30, so please add 300 loyalty points") and getting data out from the method ("What's the user's points total?").
- Write a saga that maintains consistency across services / data stores without manually setting up choreography or orchestration, with a simple try/catch statement. (A workflow method is like automatic orchestration.)
There is some limitation
Some calls in .NET do unsuspecting non-deterministic things and are easy to accidentally use. This is especially true with Tasks. Temporal requires that the deterministic TaskScheduler.Current is used, but many .NET async calls will use TaskScheduler.Default implicitly (and some analyzers even encourage this). Here are some known gotchas to avoid with .NET tasks inside of workflows:
but these limitation only applies to Temporal Workflow definition (which should be very simple in any case) and does not apply to Temporal Activities.
On handling workflow modification
Temporal determines a workflow is non-deterministic if upon code replay the high-level commands don't match up with what happened during the original code execution. In this case, technically it's safe to change the code this way because both still result timer commands. But what the timer was first created with on existing runs is what applies (there are reset approaches though as mentioned in other comment). However if, say, you changed the implementation do start a child workflow or activity before the timer, the commands would mismatch and you'd get a non-determinism error upon replay.
On Temporal workflow vs event bus
The major differences between an event bus and a workflow engine is the workflow engine:
Creates and updates events/messages for you
Maintains consistency between the events and timers and the state of the processing flow. More info on why this is important for correctness/resilience: https://youtu.be/t524U9CixZ0
The difference between workflow code and using an event bus is that with the former, the above is done automatically for you, and in the latter, it's done manually, which can be a lot of code to write and get right, and is harder to track/visualize what happened in production and debug. It would also take a lot of events to get an equivalent degree of reliability—the message processor would need to do a single step and then write the next event to the bus. So a 10-line workflow-as-code function would translate to 10 different events in the bus route.
What workflow means in Temporal
Traditional workflow systems are often used for specific types of business processes, particularly those that are async like human-in-the-loop.
Temporal, while it uses the "workflow" terminology, is a new type of thing. At a basic level, it's "do you want your backend code to run reliably?" If yes, and you're okay with the latency hit of each step getting persisted for you, then the answer is "use Temporal to write your backend code." It's a new programming model that lets you develop at a higher level of abstraction, where you don't have to be concerned about faults in the hardware or network or downstream services/3rd party APIs being temporarily down. Where you no longer have to code retries, timeouts, use task queues, or use a message bus to communicate between services. And oftentimes don't even need to use a database.
SilverKey will start using Temporal as soon as next week in our next big project after researching and creating prototype for the past 4 months.