Java multithreading results in 100% CPU usage solution and proper shutdown of thread pools _java_ Script Home
Script house Common server software
Quick navigation
Your location: Home page → Software programming → java→ Java multithreaded CPU usage is 100%

Java multithreading causes CPU usage to be 100% resolved and thread pool to be closed correctly

Updated: May 27, 2021 09:39:58 Author: levi125
10 million table data import memory database, according to page size 10000 query, multithreading, 15 threads run, and finally found that the CPU occupancy 100% card dead, then how to solve, this article to introduce, interested friends can understand

intro

Scenario: 10 million table data imported into memory database, query by page size 10000, multithreading, 15 threads running. Use the ExecutorService executor = Executors. NewFixedThreadPool (15) local ran after a period of time, found that the computer CPU increased, finally 100% CPU card dead, memory usage is as high as 80%.

Troubleshoot problems

Debug finds that although a thread pool with a fixed length of 15 is created, the List iterated in For will continue to be added to every waiting task in the LinkedBlockingQueue() queue due to the large amount of data. Therefore, whether it is CPU preemption of thread number, or memory consumption is extremely high. So you can control the upper limit of the wait queue LinkedBlockingQueue.

solution

Use AtomicLong to check whether the thread has completed, and then execute executor.submit() to submit a new task to the lead queue. The pseudo-code is as follows:

private AtomicLong threadNum = new AtomicLong(0); public void init() throws Exception { ExecutorService executor = Executors.newFixedThreadPool(15); Integer total = accountMapper.selectCount(new QueryWrapper<>()); Integer pageSize = 10000; Integer pageCount = (total + pageSize -1)/pageSize; // Page size INTEGER pagecount = (total + pagesize-1)/pagesize; // Total number of pages for (Integer start = 1; start <= pageCount; start++) { List<Account> list = accountMapper.selectPage(new Page<>(start, pageSize), query).getRecords(); // Wait for the Thread task to complete, set 30, the number of running threads is 15, the number of waiting queue threads is 15 while (threadNum.get() >= 30){thread.sleep (5000); } / / open a thread + 1 threadNum incrementAndGet (); executor.submit(() -> {try {// dealMessage(list); / / task - 1 threadNum. DecrementAndGet (); } catch (Exception e) { e.printStackTrace(); }}); } executor.shutdown(); executor.awaitTermination(1, TimeUnit.DAYS); }

The effect is to keep the CPU between 15 and 45%, and the memory usage is only 45%.

So far I can only think of such a way to control the upper limit of waiting queue LinkedBlockingQueue, there is a better way to let me know, thank you!

2021-02-03- The dividing line has recently used multi-threaded development and found that there are still many ways to control it. A simple Semaphore token flow limiting control using java is also possible.

Multi-threading:

  • The thread pool must be closed before the main thread can be finished (and the interface will return). finally {executorService.shutdown(); }
  • The main thread waits to ensure that all the subthread tasks of multiple threads have completed, and then ends. -> executorService.awaitTermination(1, TimeUnit.DAYS);
  • semaphore token flow limiting controls the fixedThread thread pool, which in this case is a maximum of 2 threads working at the same time
  • Execute () The difference between fixedThread.execute() and fixedThread.submit() is that the latter returns the result, but the latter catches the exception message and cannot be thrown to the main thread.

public static void main(String[] args) { final List<String> tableNames = new ArrayList<>(); tableNames.add("a"); tableNames.add("b"); tableNames.add("c"); tableNames.add("d"); tableNames.add("e"); tableNames.add("f"); final Semaphore semaphore = new Semaphore(2); final ExecutorService fixedThread = Executors.newCachedThreadPool(); for (final String tableName: tableNames) {// block, acquire token try {semaphore.acquire(); } catch (InterruptedException e) { e.printStackTrace(); } //do fixedThread.execute(() -> { //can throw ex log final ExecutorService executorService = Executors.newCachedThreadPool(); try { executorService.submit(() -> { //can't throw ex log //int i = 1/0; System.out.println("tableName2:" + tableName); }); //int i = 1/0; System.out.println("tableName:" + tableName); } catch (Exception e) { e.printStackTrace(); } finally { executorService.shutdown(); try { executorService.awaitTermination(1, TimeUnit.DAYS); } catch (InterruptedException e) { e.printStackTrace(); } semaphore.release(); System.out.println("semaphore.release"); }}); } // Remember to close the thread pool fixedThread.shutdown(); try { fixedThread.awaitTermination(1, TimeUnit.DAYS); } catch (InterruptedException e) { e.printStackTrace(); } System.out.println(" Main thread...") ); }

Print result

tableName:b tableName2:b tableName:a tableName2:a semaphore.release semaphore.release tableName:d tableName2:d tableName:c semaphore.release tableName:e tableName2:c semaphore.release tableName:f tableName2:e semaphore.release tableName2:f semaphore.release Main thread...

To this article on Java multithreading caused by the CPU occupancy 100% solution and thread pool the correct way to close the article is introduced to this, more related Java multithreading CPU occupancy 100% content please search the previous articles of the script home or continue to browse the following related articles hope that you will support the script home in the future!

Articles you may be interested in:

Related article

Latest comments