threadpool – Page 2

multiprocessing.Pool – PicklingError: Can’t pickle : attribute lookup thread.lock failed

July 25, 2023 by Tarik

multiprocessing passes tasks (which include check_one and data) to the worker processes through a mp.SimpleQueue. Unlike Queue.Queues, everything put in the mp.SimpleQueue must be pickable. Queue.Queues are not pickable: import multiprocessing as mp import Queue def foo(queue): pass pool=mp.Pool() q=Queue.Queue() pool.map(foo,(q,)) yields this exception: UnpickleableError: Cannot pickle <type ‘thread.lock’> objects Your data includes packages, which … Read more

Thread.Start() versus ThreadPool.QueueUserWorkItem()

July 24, 2023 by Tarik

Starting a new thread can be a very expensive operation. The thread pool reuses threads and thus amortizes the cost. Unless you need a dedicated thread, the thread pool is the recommended way to go. By using a dedicated thread you have more control over thread specific attributes such as priority, culture and so forth. … Read more

Ensuring task execution order in ThreadPool

July 20, 2023 by Tarik

Something like the following will allow serial and parallel tasks to be queued, where serial tasks will be executed one after the other, and parallel tasks will be executed in any order, but in parallel. This gives you the ability to serialize tasks where necessary, also have parallel tasks, but do this as tasks are … Read more

Is it really my job to clean up ThreadLocal resources when classes have been exposed to a thread pool?

July 17, 2023 by Tarik

Sigh, this is old news Well, a bit late to the party on this one. In October 2007, Josh Bloch (co-author of java.lang.ThreadLocal along with Doug Lea) wrote: “The use of thread pools demands extreme care. Sloppy use of thread pools in combination with sloppy use of thread locals can cause unintended object retention, as … Read more

How can I shutdown Spring task executor/scheduler pools before all other beans in the web app are destroyed?

July 8, 2023 by Tarik

Two ways: Have a bean implement ApplicationListener<ContextClosedEvent>. onApplicationEvent() will get called before the context and all the beans are destroyed. Have a bean implement Lifecycle or SmartLifecycle. stop() will get called before the context and all the beans are destroyed. Either way you can shut down the task stuff before the bean destroying mechanism takes … Read more

Python 3: does Pool keep the original order of data passed to map?

July 2, 2023 by Tarik

Pool.map results are ordered. If you need order, great; if you don’t, Pool.imap_unordered may be a useful optimization. Note that while the order in which you receive the results from Pool.map is fixed, the order in which they are computed is arbitrary.

How Threadpool re-use Threads and how it works

June 13, 2023 by Tarik

If there is no need to create new Thread in ThreadPool scenario, then how it works with same thread which just finished its run method, will that Thread can be used again? Simple – the original thread never actually completes. It just waits for another task to execute. In pseudo-code: // No, this isn’t even … Read more

What determines the number of threads a Java ForkJoinPool creates?

June 13, 2023 by Tarik

There’re related questions on stackoverflow: ForkJoinPool stalls during invokeAll/join ForkJoinPool seems to waste a thread I made a runnable stripped down version of what is happening (jvm arguments i used: -Xms256m -Xmx1024m -Xss8m): import java.util.ArrayList; import java.util.List; import java.util.concurrent.ForkJoinPool; import java.util.concurrent.RecursiveAction; import java.util.concurrent.RecursiveTask; import java.util.concurrent.TimeUnit; public class Test1 { private static ForkJoinPool pool = new … Read more

How are Threads allocated to handle Servlet request?

June 9, 2023 by Tarik

Per request means when an HTTP request is made, a thread is created or retrieved from a pool to serve it. One thread serves the whole request. Thread per connection would be the same thing except the thread is used for an entire connection, which could be multiple requests and could also have a lot … Read more

Not much difference between ASP.NET Core sync and async controller actions

June 7, 2023 by Tarik

Yes, you are missing the fact that async is not about speed, and is only slightly related to the concept of requests per second. Async does one thing and only one thing. If a task is being awaited, and that task does not involve CPU-bound work, and as a result, the thread becomes idle, then, … Read more