What is parallelism

Parallelism focuses on:

Whether multiple tasks are actually executed at the same time at the same moment.

It leans more toward performance and throughput, rather than mere program structure.

1.An intuitive understanding

If a task can be split into several independent jobs:

任务 A
任务 B
任务 C
任务 D

and the machine has multiple CPU cores, then these tasks may run simultaneously.

This is parallelism:

多个执行单元
同一时刻
同时干活

2.The difference between parallelism and concurrency

Concurrency is:

多个任务都在推进

Parallel is:

多个任务真的同时执行

So a program can:

• Concurrency without parallelism: multiple threads execute in turn on a single-core CPU;
• Concurrency with parallelism: multiple threads compute simultaneously on a multi-core CPU;
• Use parallel algorithms, but do not manually create threads: let the standard library or the underlying runtime decide how to execute.

3.Parallelism does not mean that the more threads, the faster.

When the number of threads exceeds the number that the hardware can execute simultaneously, the operating system must continuously switch threads.

Thread switching, synchronization, lock contention, and cache invalidation all introduce additional overhead.

Therefore, when optimizing in parallel, pay attention to:

1. Can the tasks be split apart?
2. After splitting, are they sufficiently independent?
3. Is the computational load of each task large enough?
4. Do the tasks frequently compete for the same data?
5. Do the benefits brought by parallelism outweigh the scheduling and synchronization costs?

4.C++ Parallel content that may be encountered

The following can be added later:

• Standard library parallel algorithms;
• execution policy;
• Task parallelism;
• Data parallelism;
• SIMD and vectorization;
• Thread pools and task scheduling.

For now, let's reserve this spot; it will be more appropriate to expand on it after finishing the fundamentals of concurrency.