How can I manage two tasks that each take more than two hours to execute in .net MVC? I've been fetching a huge amount of data from a third-party website, and although I've tried using async/await, there's no improvement since the issue lies in fetching such large data and then performing calculations and storing it in a database. How can I move these tasks to a background thread to prevent the application from freezing?
Code or reference documents related to that.
There are many ways to do that, from the primitive Thread class, over Task to a full-fledged separate background process.
Since the OP is tagged with the backgroundworker tag, a word about the BackgroundWorker class may also be in order. As far as I recall, this component has been around forever, and is mostly intended for use in rich client applications (Windows Forms, possibly WPF). The System.ComponentModel namespace also suggests that this isn't intended for web use.
You may instead want to read Microsoft's documentation Background tasks with hosted services in ASP.NET Core.
Which of all these options is best depends on the overall requirements for the application - both architecturally and related to resilience, observability, and many other concerns. Do you need to display a progress bar (or similar) for the long-running process? Should you enable a user to cancel the job? What should happen if the long-running process crashes after running for an hour? What happens if the web server restarts while the long-running process runs? What should happen if a new one long-running process is started while one is already running?
Depending on the answers to such questions, one technology may suggest itself as more appropriate than the others.
