Here we have it. Chinese AI hardware

[TailsWin]

The Houmo M50 chip has been cooking for a while so I'm not sure if it's only now out, or it just now got packaged as a product. Either way, I presume the RAM is Chinese too, since they do make DDR5.

Overall it's close to what I expected for a first gen product: simple NPU and RAM, on a M.2 board (SSD size), 15W power consumption compared to... Well, hundreds of W for a GPU.



Reddit is making fun of it for having slow RAM. I guess that's valid, but it's clear that it's 1) not a very optimized product (I still expect someone will make good of the idea of just bolting tensor cores directly onto RAM chips rather than have a separate chip, as this will always be slow), 2) it's not actually competition for GPUs, more like for edge devices, DIY or robots, for which this could be great. They benchmark 8B and A3B models at 25 tk/s which isn't much if you want to refactor your leaked Windows Vista codebase, but it should be able to run a couple 1B-3B models in parallel doing all kinds of things, like video recognition with transcribing and simultaneous search or whatever.

Cost is apparently about $1200 for 24GB, which sounds expensive, but I dunno. Maybe that's just prices for outsiders, or first batch cost, or they see it as specialised hardware. So I guess in that context, it makes sense.

We've been here before, with so many other high tech products. Once the Chinese start pushing, within 5 years we should see 1/10 of the cost and 10x the power, unless something blows up.

Obv these guys aren't the only ones with their own hardware, tons of other companies have their chips, including Xiaomi. But this is something you can actually buy and plug into a PC, at least in theory.

BTW M.2 AI cards have also existed for a while, but nothing actually serious for real use, more like funny toys for research, simple robotics and such.

 

[Otto Von Herunterhängen]

With the profits currently being made on supplying the hardware for AI, you have to believe others are trying to crash the parties, and it helps if your government takes a loose view of other people's IP rights (not that I don't think the IP laws need changing).

I'm wondering how much of a market there would be for this in PC's though. I mean, what the hell are you supposed to do with it?

 

[TailsWin]

With the profits currently being made on supplying the hardware for AI, you have to believe others are trying to crash the parties, and it helps if your government takes a loose view of other people's IP rights (not that I don't think the IP laws need changing).

I rather think that it's a gen 1 product for industrial apps rather than end-users, so in that sense the price is sensible. The lower power consumption alone can make it worthwhile over running a full gpu. "AI boards" with just some ARM cpu cost not much less, even if they're essentially just phone parts.

Re IP, China isn't all that dependent on IP theft anymore, at least not in this industry. Their DRAM production is their own with some old patents bought together with a closed German producer, NPUs are actually pretty simple so any decent designer can cook them up, and in AI itself China is leading the research, if anything. DeepSeek, Kimi and Qwen are always at the forefront of almost every new AI innovation, and they make all of it public. So at this point I'd rather expect the US companies stealing from them, not the other way around. Anyone can do AI research right now, it's not rocket science. That's the main reason it exploded so quickly.

I'm reminded of situation in high end DACs and other audio equipment like professional headphones and in-ear monitors. Even after the collapse of consumer audio, the high end stuff was expensive and exclusive. Then the Chinese got into it, first awkwardly copying some designs which everyone was mocking, then they started doing their own stuff and within 5 years, you could get a $100 DACs and $100 monitors with better quality than what would've cost 5x as much a few years earlier.

I think this will be similar. If you look at that M2 board, it's just off the shelf chips (well, the NPU is their own chip but still, it's standalone) hooked together, which is inefficient. It's the early awkward stuff, and yet already something the western companies don't want to do because it could cannibalise their cash cows.

I've said before that with the tech that exists now, you could squeeze a 5x as powerful board onto an SD card if you optimised the design. But it has to come from new people, not the old guard.

I'm wondering how much of a market there would be for this in PC's though. I mean, what the hell are you supposed to do with it?

You can run a bunch of small AIs without using your CPU or GPU. Assistant, translation, transcription, OCR, captioning, smart home, camera surveillance with object recognition, or whatever. There's literally half a million AI models under 12B for download on HF. For now it's a niche thing mostly for embedded devices, but once this stuff gets more powerful, it should be able to run serious coding models or video generation or whatever, at 3% of power usage of GPUs.