If you have been following the semiconductor market recently, you have probably noticed that the share prices of Samsung Electronics and SK hynix are being influenced by much more than their latest earnings. HBM supply, investment in AI data centers, memory prices, and the release schedules of NVIDIA's next-generation AI accelerators can all move semiconductor stocks significantly.
Even when a semiconductor company reports strong results, its share price may fall if investors begin to question whether that growth can continue. On the other hand, expectations of sustained AI investment can quickly lift semiconductor stocks again. The market is no longer looking only at whether memory prices are rising. Investors are also trying to determine which technologies could improve AI system performance after HBM. One term that is gradually appearing in these discussions is PIM.
What PIM Actually Means
PIM stands for Processing-in-Memory. In simple terms, it refers to memory that can perform some computational work on its own. It is sometimes loosely described as in-memory computing, but the two expressions are not exactly interchangeable. Traditional in-memory computing generally means placing data in system memory rather than slower storage so that applications can access it more quickly. PIM goes a step further by adding processing capabilities inside, or very close to, the memory itself.
Why Data Movement Has Become a Bottleneck
In a conventional computer, the CPU or GPU performs calculations while memory stores the data needed for those calculations. The processor retrieves data from memory, works on it, and then sends the result back. You could compare this to taking ingredients out of a refrigerator, carrying them to a kitchen counter, preparing a meal, and then moving the finished food somewhere else for storage.
That process does not seem particularly inefficient when it happens only a few times. The problem becomes much more noticeable when billions of pieces of data must be moved repeatedly. At that scale, carrying the data back and forth can consume a significant amount of time and electricity.
Artificial intelligence systems face exactly this problem. Generative AI and large language models repeatedly read and process enormous volumes of data. Even if a GPU can perform calculations extremely quickly, it may still have to wait if memory cannot deliver the required data fast enough. Processor performance has improved rapidly, but the speed of transferring data between the processor and memory has not always kept pace. This limitation is commonly known as the memory bottleneck or memory wall.
How HBM and PIM Solve Different Problems
HBM, or High Bandwidth Memory, has become one of the most important technologies for addressing this issue. HBM stacks multiple memory dies vertically and provides much wider data pathways than conventional memory. If ordinary memory is like a narrow road, HBM is more like a multi-lane expressway. It can deliver far more data to a GPU at the same time, which is why it has become a critical component of modern AI accelerators.
This also explains why the HBM capabilities of Samsung Electronics and SK hynix are so closely watched by investors. NVIDIA and other AI chip companies need large quantities of high-performance memory, and the ability to manufacture reliable HBM products has become a major source of competitiveness in the semiconductor industry.
PIM takes the idea one step further. Instead of moving every piece of data to a separate processor, it allows certain simple and repetitive calculations to be performed where the data is already stored. Returning to the warehouse analogy, PIM is like installing a small workstation inside the warehouse instead of sending every item to an outside facility for processing.
PIM is not intended to handle every type of calculation. However, it can take care of selected operations that frequently appear in AI workloads, including repeated multiplication and addition. Performing these tasks close to the memory can reduce the amount of data that must travel between the GPU and memory.
Reducing data movement can improve both processing speed and energy efficiency. In an AI data center, electricity consumption and cooling costs are now almost as important as the raw performance of the chips themselves. When tens or hundreds of thousands of accelerators are running together, even a small improvement in power efficiency can make a substantial difference to total operating costs.
For this reason, semiconductor architecture is gradually shifting from simply moving data faster to avoiding unnecessary data movement altogether. HBM expands the road between memory and the processor, while PIM attempts to reduce the number of trips that must be made on that road.
Where PIM Could Be Used
PIM could eventually be used in generative AI training and inference, recommendation systems, speech recognition, image processing, search services, and high-performance computing. It is particularly promising for workloads that repeat similar calculations across very large amounts of data.
The inference stage may become especially important. Training an AI model requires enormous computing resources, but once the model has been trained, it must respond efficiently to requests from millions of users. If PIM can help process those requests with lower latency and reduced power consumption, it could become a valuable part of future AI infrastructure.
The same idea may also become relevant to on-device AI. As more AI functions are performed directly on smartphones, automobiles, computers, and other edge devices, manufacturers will need to balance performance with strict power limitations. A technology that reduces data movement and electricity consumption could be useful in environments where large data-center GPUs cannot be used.
PIM is not expected to replace CPUs or GPUs. CPUs and GPUs will still be responsible for complex calculations, overall system control, and tasks that require flexible decision-making. PIM is better understood as a supporting technology that handles selected operations responsible for heavy data movement.
Software is another important part of the equation. Developers need tools that can determine which calculations should be assigned to PIM and which should remain on the CPU or GPU. Existing AI software must also be adapted so that it can take advantage of PIM hardware. Without a suitable programming environment and a broader software ecosystem, even an impressive semiconductor design may struggle to achieve widespread adoption.
There are technical challenges as well. Memory chips are designed to store large amounts of data reliably, while logic chips are designed to perform complex calculations quickly. Combining these different functions can increase manufacturing costs and design complexity. Engineers must also consider heat, production yield, reliability, and the amount of space that processing units occupy inside the memory.
The performance improvement offered by PIM will not be identical across every application. Some workloads are limited primarily by data movement and may benefit greatly, while others depend more heavily on complex processing and may see a smaller advantage. Figures announced by semiconductor manufacturers therefore need to be understood in the context of the specific workloads and system configurations used for testing.
Why PIM Matters to the Semiconductor Market
Samsung Electronics has introduced HBM-PIM, which combines HBM with AI processing capabilities, and has explored its use in AI accelerators and high-performance computing systems. SK hynix has also developed GDDR6-AiM, which adds processing functions to GDDR6 memory, along with the AiMX accelerator card that incorporates multiple GDDR6-AiM chips.
The two companies are taking somewhat different approaches, but the underlying direction is similar. Both are attempting to transform memory from a passive component that merely stores data into an active component that participates in AI computation.
This is also something to remember when the expression “PIM-related stocks” begins appearing in financial news or online searches. A company's involvement in PIM research does not necessarily mean that the technology is already generating significant revenue. At present, the earnings of Samsung Electronics and SK hynix are affected more directly by factors such as HBM shipments, server DRAM demand, memory prices, customer qualification, manufacturing capacity, and capital expenditure.
PIM is better viewed as a longer-term technology that could help memory manufacturers maintain their competitiveness after the current HBM expansion cycle. It may become an important growth opportunity, but a PIM announcement alone should not be treated as evidence that a company's sales or share price will immediately rise.
One reason semiconductor stocks can fluctuate even after strong earnings is that the market is always looking several years ahead. Investors are asking whether investment in AI data centers will continue, whether massive capital spending will produce sustainable returns, and how quickly Chinese memory manufacturers will catch up. Expectations surrounding these questions can sometimes have a greater effect on share prices than the latest quarterly results.
Within this broader picture, PIM should be considered one of several technologies being developed to improve both the performance and power efficiency of AI systems. It is not yet accurate to say that PIM has entered widespread commercial adoption. However, as AI models process increasingly large amounts of data, the boundary between memory and processors is likely to become less distinct.
If HBM is a technology that widens the road between memory and the GPU, PIM is a technology that places a small processing facility where the data is stored. That is probably the simplest way to understand the difference.
When reading semiconductor news in the future, it may be helpful to look beyond HBM and become familiar with terms such as PIM, AiM, CXL, and PNM. Understanding these technologies makes it easier to see how the AI semiconductor market is evolving and why memory is becoming much more than a component that simply stores data.
This article is also available in Korean: Read the Korean version