Chinese AI chips fall short on coding, forcing firms to stretch scarce Nvidia supply

Chinese AI companies are optimising software to cope with surging demand for inference, as part of that workload still relies on computing power from a limited pool of high-end chips amid restricted access to Nvidia processors.Compared with training an Artificial Intelligence model, which relies on high-end chips, inference – a later phase in which the trained model applies its knowledge to process responses – can be adapted to domestic hardware. However, industry insiders said complex tasks like coding still required Nvidia chips, which meant the sector was facing acute compute constraints as AI moved from model development to large-scale deployment.“The demand side is now showing a bipolarisation,” said Guan Jiawei, vice-president of inference optimisation start-up Approaching.AI, noting that demand for high-quality tokens – the basic units of data that models process and generate – far outstripped supply.