In the previous post I’ve written about why I think the ability to schedule tasks for later execution is a fundamental technical feature, but also a must have from business’ point of view. We are passed the whys, so let’s get to the hows. The answer is simple - Hangfire. I’ve written about it here, here and here, so yeah, you guessed it, I like it. Hangfire is an amazing library. It has shown it’s value in my pet project (cookit.pl) and in a huge ERP system that we are building at work, where we replaced Quartz.NET with it and never looked back.Continue reading...
This is a post about the importance of not doing stuff right here and right now. About doing stuff later, delaying them and in a perfect world making someone else do them. But I will still stick to IT, don’t expect any life tips or guidance.
A mindset I see constantly in system design is doing the right thing right now, and even more stuff after that. Like sending an email, doing a REST call, performing some CPU intense calculations, calling an external resource. We are doing those things where they should be from the logical point of view. So, if we are given a task like registering an user that sound more or less like that:Continue reading...
This is a rectification for the previous post about the cost of garbage collection. If You didn’t read it give it a try and check if You can spot the bug/mistake.
Like Konrad pointed out in his comment not all objects were in generation 0 as I assumed. This is partly connected to the fact that .NET, seeing rapid need for memory in
Setup, will try to collect some of them, calling garbage collection, but I make the matters worse by calling
GC.Collect in the end. So making sure that objects created during
Setup will be in generation 2 (in my defence it was left over after a bit different take on this problem).
This is the proper code (it also was committed to the github repo):Continue reading...
Update There is a error in the first version of this code, and the tests are in fact always operating in a situation that almost all object are in generation 2. The answer is in the next post
In the previous post I’ve promised to write how to have differentiate the number of workers in Hangfie, but in the comments Michał asked one interesting question - “Is generation 1 collection more expensive then generation 0 collection?”. Going further how does generation 2 collection fit into it?Continue reading...
One of the steps in cookit is calculating similar recipes. This is what you can see on the left on the recipe page like this
For the sake of clarity and manageability it’s scheduled as separate Hangfire jobs. Because cookit is running 5 workers, so similarities are calculated for 5 websites concurrently.
The process uses cosine similarity, so it allocates a huge list at start and calculates similarities. A very CPU heavy operation.