First Experiences with a Copilot+ PC

Introduction

In Sept 2024, I recieved my new toy, a Lenovo Yoga 7x Slim Copilot+ PC 32GB Model with the intention of using the onboard powerful NPU for running AI models locally. This was a fairly big jump to move away from Intel based devices to ARM64 with the initial expectation there would be teething issues with application compatibility. Well with the apps that I care about, and I have been plesantly suprised as I will go into in more detail, I have not yet encountered difficulty in terms of compatibility YET.

Other devices I had considered, these are very nice devices, each with their own merits and issues:

• Surface Laptop 7
• Dell Latitude 7355
• Dell XPS 13

For me balancing cost, features, and value were the deciding factors and wanted to limit the risk for me to switch to an ARM64 device, to give me time to build confidence in the architecture. I have seen after purchasing that Intel and AMD have released their versions supporting Copilot+ PCs so will be interested in how they compare.

My developer rig and what I have installed

I primarily use my laptop for community work, development and research for new technology, so I dont run every package and technology under the sun, but here is my new rig and what I use to develop and research:

Majority of my development is done via the WSL Linux with NVM installed to switch node versions. I had considered installing the latest Node version v20 as that has ARM64 support, BUT frameworks such as SPFx do not yet support this version of node - for me its not a deal breaker its not stopping development.

NVM doesnt yet support ARM64 to install specific versions of node with this architecture but I have seen a few folks ask teh owner to add this capability.

I haven’t finished setting up, but these are my core apps. I tend to install only as needed rather than fill my laptop with programs I might use.

The NPU - Trying to Update the driver from Qualcomm

So this one is fun, to get an application to use this type of processor is less that straighforward. When I got the device, I tried to update to the abolsute latest Qualcomm driver especially since the announcement came out about the updates to the DirectML to support Copilot+ PCs. So I proceeded to remove the existing driver which disables the NPU, I then found the new driver does not specially support my model laptop, after reading the failed install logs YAY! (OK, I was not that calm lol)

Uh oh:

I proceeded to restore the NPU driver and found that Lenovo support had not yet recieved the driver from the factory. LMAO, oh damn, how do I get this back and queue System Restore point, which crashed half way through restore, and needed proceeded with automatic repair. Of course, I have Windows 11 Pro so it has bit locker, and needed my recovery key - which I didnt backup, BUT i did use my 365 tenant to login, so LUCKILY found the key in Entra ID (Phew, Phew, Phew). So eventually managed to recover THANK YOU, lol at this point only had it for a day.

The moral of the story, if you are going to “play” with drivers:

• Create a system restore point
• Backup PC if you feel that is needed
• See if the original drivers are available
• Ensure your bitlocker recovery keys are recoreded OFF device

At this time, the Copilot+ PCs are very new, software is catching up, but not fully ready in some areas, expect issues.

Qualcomm AI Hub

I havent used the device long but I have been exploring the Qualcomm AI Hub, this contains a rich repository of models for the Snapdragon Elite X, beyond generative AI, so if you are looking to perform other tasks such as transcribing, image generation, image analysis, generative AI (Llama) and more to utilise the devices this is the place to start.

Interestingly, their approach is to perform model optimisation for the device architecure in their Cloud and then run on-device locally. This is a fantastic resource and I definietly not doing it justice, its a steep learning curve, but a fun one.

I am spending alot of time here because I want to leverage my NPU and this is the place to be, and trying to learn Python at the same time but I really want this to work with the dotnet ecosystem, and I would like to see if their model optimation tools can process Phi 3.5 models too.

What about hardware compatbility

I have had some small hardware issues with accessories such as my Dell Thunderbolt dock, in fact my wife’s new setup works better with this device that my own (of course it does lol).

My Stream Deck, Logitech cameras, Keyboard, Mice, External storage work as expected. I am happy with the overall compatibility, as I prefer to roam around the house, and the screen on this device is super nice!

One thing I want to call out (in a good way), this thing is SILENT, FAST and BATTERY is super awesome. I don’t know if the fan has ever turned on, the device feels cold, this laptop does not produce anywhere near the heat my Dell XPS (9th Gen Intel) and its aircraft level noise the fan produced.

Conclusion

What I have found, is there is a lot of hype and a TON of work going into enabling these devices to leverage the NPU. I dont feel all the software, drivers are fully baked, but given the pace of change in this space in the industry, I am confident that its a matter of patience before these capabilities are available.

As for ARM64, I am really happy with it, if its not ARM64 supported, the Microsoft Prism emulator works really well for my purposes and I have seen that software vendors are growing support for ARM64 compiled devices. Its not a blocker for me.

I have a long and FUN journey ahead to fully leverage the capabilities of the NPU, and I have more to share around specifically the Copilot+ PC part and experiences of ARM64 devices.

Resources

Here are some cool resources:

• Qualcomm AI Hub
• Lenovo Store
• Development for Copilot+ PCs

Enjoy!