Chrisoft

This review came at the very least 6 months too late.

Rationale for a new laptop

My old trusty first gen ThinkPad X1 yoga is showing its age: its U-tier ultra-low voltage processor is having trouble keeping up with the “modern web” that is constantly putting increasing pressure on the browser. Its integrated GPU can barely handle Minecraft at 60 fps, not to mention hundreds of virtual instruments I mess with are struggling with the processing power of this machine which wasn’t anything to brag about even when it first came out. It also took a lot of beatings, which are mainly from my own abuse, like this one (warning: post is in Chinese), and is nearly falling apart. Considering I’m not well settled in this land that is alien to me, a full fledged desktop is hard to justify. So a new laptop it is.

Additionally, unlike their 11th gen desktop processors, Intel’s 11th gen ultrabook offerings looks extremely compelling due to the new Xe integrated graphics introduced in this generation. Its (theoretical) performance is finally catching up with previous generation entry-level gaming graphics and current generation dedicated graphics found in slim laptops (1050 Ti and MX 350/450). Of course this is Intel and theoretical performance doesn’t mean much when their driver kind of sucks (severe performance drops and incompatibilities in some games). But it’s been rock solid for daily desktop usage (and Minecraft gaming because that’s literally the only game I play now). So I decided I’m just going to roll with it.

Why Framework?

I’m not a fan of laptops in the desktop replacement category, even I often use mine as one (in terms of the workload I run on it). They are just too chunky for liking.

Trackpoint is essential for my survival when using a laptop. Instead with torturing myself with a huge touchpad, I’d rather stick with an old ThinkPad with no touchpad.

Wait, don’t these two criteria alone restrict my choice to ThinkPads and a few other business laptops?

Well, true… but when I laid my hands on a 9th generation X1 Carbon, it feels so different that I wasn’t even convinced that I was interacting with a ThinkPad anymore: especially the new keyboard with further reduced travel and the trackpoint buttons with virtually no tactile response. I decided to extend my radar to include models that don’t have a trackpoint, and use an external trackpoint keyboard with it if I get one of such models instead. Of course I was not ruling out ThinkPads, I was just allowing myself to get something other than a ThinkPad.

In Feburary 2021 I saw a few news articles about this new “Framework” thing popping up. I find it interesting, but the modular expansion thing is just a gimmick, and I had suspicion that they are trying to ride the hype around right to repair – not to say that is wrong. Of course the hardware configuration is all fine and dandy, but I’m no buying something with a single selling point of expansion modules (which at the time of writing offers nothing but ports and features found on almost every single laptop on the market).

When further details on the laptop surfaced, it started to get interesting. It was suggested that motherboard of future revisions can act as drop-in upgrades, and that the motherboard can be easily transformed into a small form-factor PC. Also display and keyboard assembly would be available for consumers to purchase online in case a replacement is necessary. This immediately bring this laptop to a level higher in my tier list, as it was becoming apparent that they are doing actual work.

And then just a few weeks after I got into the States, my X1 yoga started experiencing random system-wide freezes once again. It was a good excuse for a new laptop, I thought. I checked Lenovo’s website but found out (pretty shockingly) that there was no discount active during that time, and all X1s were ludicrously priced. And then I placed my pre-order of the Framework Laptop. (Of course I ordered the DIY edition, saved me a decent chunk of funds!)

Important dates

• Pre-order placed: 2021-09-16, Batch 4 (I swear this is totally unrelated to Linus Sebastian’s Framework investment video)
• Remaining balance charged: 2021-10-06
• Shipped: 2021-10-11
• Received: 2021-10-13 (probably leaks my rough geological location, but whatever)

Configuration ordered

• DIY edition
• i7-1185G7
• 64 GB RAM (2x Crucial CT32G4SFD832A shipped)
• No SSD (Bring your own). 1 TB Samsung 970 Pro installed.
• Intel AX210 wireless
• ANSI keyboard (US layout)
• 2x USB Type-A expansion modules, 4x USB Type-C expansion modules, and 1x HDMI expansion module
• With power adapter
• Windows 10 Professional license
• $2153 before tax + $230 for the SSD

Unboxing and setting up

The laptop arrived in an inconspicuous shipping box, with symbols on it suggesting it contains stuff that may explode. Inside is the actual box with intriguing designs. There was only minimal amount of paperwork in the package.

Despite being the DIY edition, the laptop comes almost completely pre-assembled. There’s a blog post on the official website explaining why that is the case. For this reason putting the thing together is incredibly straight forward. Within minutes I got the thing up and running (while booting into nothing of course).

The box	Unboxing
The box inside the box	Paperwork
Internals	Components in place

I decided to get Windows working first, even it hasn’t been my primary operating system for almost a decade. I prepared the first release of Windows 11 on a USB stick. But of course I decided to torture myself by not using the official installation media (it doesn’t matter anyway, as after the first attempt failed I tried the official installation media and it also failed for some reason, see the pictures below). Eventually I got the dism installation procedure working.

Windows, being Windows, gave me a few silly errors when I was going through the OOBE. The official driver package is necessary to get all devices working.

Next step will be my daily driving operating system. Of course it’s going to be Gentoo. What do you mean I’m insane?

I used the Arch Linux ISO to bootstrap the installation. Now I made people from both distros hate me.

After roughly 10 hours of CPU stress testing, a basic KDE plasma desktop was emerged. Not too bad for a slim laptop.

Trigger warning

First neofetch

Performance

For a comprehensive performance review of the Framework laptop, you should be reading a more professional review, not this article. This section will focus on my personal use cases instead.

Processing power

My daily work, when it comes to the processing power intense parts, mainly involves running tests, building small-to-moderate sized projects, and performance profiling. Of course there is also routine rebuilds of huge applications due to my choice of distro. Plus some music mixing when I have the time for it.

Thanks to the two additional cores and drastically improved clocks and IPC, the 11th gen i7 is more than 3 times more powerful than my old 6th gen i7 for code compilation workloads, while having similar sustained power consumption under such workload (RAPL reports ~28 watts for both machines). As an example, Chromium 98.0.4710.4 took 883 minutes to emerge on my old i7, while on the Framework it took 273 minutes. The new processor is also able to handle music projects with over 100 tracks, even with a decent performance headroom. In some other cases where the load is less than “all-cores”, it even had more than 4 times the performance of its older counterpart. To achieve an even better result the only option is to go AMD, which they hadn’t yet offer [1]. Overall the performance is a win for me.

Graphics

As mentioned above, the performance of the integrated Xe graphics is finally catching up with entry-level gaming GPUs within the last 2 generations. While the Linux Mesa driver has been rock solid for desktop usage for quite some time, a lot more could be expected for gaming as online benchmark results show wild variations among different titles. Not to mention the erroneous behavior with certain aspects of legacy rendering APIs. Modern APIs behave a lot better in general.

On real-world performance, for the first time in history, Minecraft with ultra shaders (BSL) can run (correctly!) on one of my laptops with a stable 60 fps. They wouldn’t even apply correctly on any of the older Intel GPUs I’ve used. With compromises, Microsoft Flight Simulator runs (with Valve’s Proton) at a stable-ish 30 fps with acceptable visuals.

Thermal & Emissions

One word: bad.

Not intolerable, but pretty bad.

Maybe this is just the inescapable curse of all ultrabooks. But under heavy load the upper region of the keyboard is almost too hot to touch. To make things worse, the unibody-inspired design (where the gaps between keys is part of the top cover of the machine’s lower half, rather than part of the keyboard assembly) means the metal part that is between keys are at a even higher temperature. It’s not exactly enjoyable to type on the built-in keyboard when the processor is under heavy load.

Under constant max load, the processor temperature maintains steadily at around 100 degrees Celcius. This is considerably higher than my X1 yoga, which peaks at 95 degrees and drops to 85 degrees if the load is sustained.

The fan noise is also way worse on the Framework. The fan curve is quite aggressive, ramping up rapidly when the processor temperature reaches 60 degrees, and maxes out at 80 degrees and beyond. The noise is audible even in a moderately noisy room. In contrast the fan in my X1 ramps up much slower with more steps.

Also there’s no manual undervolting on tigerlake-U platforms, likely as a mitigation to the plundervolt vulnerability.

Others

One of the last consumer MLC SSDs, Samsung 970 Pro, performs well in benchmarks.

Keyboard is not great (see below). Trackpad is … meh. I don’t use that thing anyway.

The best consumer grade PCIe 3.0 SSD in the west (asterisk)

Awkward arrow key layout

ACPI reports a 8-hour estimate on battery. Running on battery for 90 minutes with light-to-moderate usage usually uses 20% of the capacity (screen always on, wireless on, typing with browser running in background), give or take, consistent with the estimation. I’ve yet to run the battery until it’s flat.

The screen is fabulous. Way more vibrant than that LCD on my first generation X1 yoga. Not a professional graphic designer or in any other job that depends on an accurate display, so that’s just my personal opinion.

Speaker is at least 10 times better than any ThinkPads I’ve ever used. Still not on par with an Apple laptop though.

Issues

Instabilities

The system sometimes goes into a complete lockup, unresponsive to any inputs (except a forceful power off), when it has been left idle for an extended period. This never happens if the system is loaded. I could not obtain any useful diagnostics.

With the release of BIOS version 3.09, which claims to resolve an “issue where the system will sometimes abnormally shutdown unexpectedly”, hopefully this such freezes will be gone. However I haven’t been running 3.09/3.10 for long enough to come to such conclusion.

Port interference? …

My ThinkPad wireless mouse suffers from stuttery movement when used on the Framework laptop. I’ve never had similar issues with this mouse before. This affects both operating systems installed. It doesn’t happen all the time, the chance of it happening is higher when the system is more heavily loaded.

Occasionally, the ports on the right side stop working altogether, and the kernel message gets flooded with output like this:

[75512.894957] usb usb2-port2: Cannot enable. Maybe the USB cable is bad?
[75517.158969] usb usb2-port2: Cannot enable. Maybe the USB cable is bad?
[75521.422973] usb usb2-port2: Cannot enable. Maybe the USB cable is bad?
[75525.686857] usb usb2-port2: Cannot enable. Maybe the USB cable is bad?
[75529.950973] usb usb2-port2: Cannot enable. Maybe the USB cable is bad?

On Windows, a notification “USB Device not recognized” is repeatedly displayed.

This knowledge base article is not relevant to my particular case, as my unit doesn’t have the problematic EMI shield.

However I’ve noticed that this issue hasn’t reoccurred for a while. Perhaps It could be a BIOS issue that has been patched since.

… and a general lack of ports

But this one can’t be resolved by a BIOS patch. With the laptop plugged in, I have a whopping 3 USB ports for USB peripherals. It reduces to 2 when I use it with the trackpoint keyboard (which is pretty much all the time as I need a trackpoint to survive). This is nowhere close to enough – considering my old X1 also has 3 USB ports, not to mention it has dedicated ports for two external monitors.

This is why I maintain my opinion that the expansion modules are currently a gimmick. There are no expansion cards that are unique enough to justify them – no WWAN card, no full size SD card reader, nor even an ethernet port (until very recently). Full size DisplayPort and storage expansion are cool, but they take up precious USB ports which means for my use case a dock is mandatory anyway if one of those cards is installed, nullifying the point of using these cards. I personally would favor a design with more fixed ports (which is already the case for the audio jack on the Framework), maybe 3 fixed USB Type-C ports on the left and 2 expansion card slots on the right. More useful expansion cards could also be a solution, like a mini hub with two USB Type-C ports. Even a proprietary port for connecting expansion cards to the computer (like two sets of USB 4 signals in a single port) is acceptable for me, to allow simpler designs of some dual function cards (e.g. storage card with a USB port); as these on board connectors aren’t meant to be used directly, users are always expected to plug an expansion card into them.

Keyboard

Like many people coming from a ThinkPad with 1.8 mm key travel, I find the keyboard of the Framework laptop lacking in tactile response. Even when compared to the keyboard on a 9th gen X1 Carbon which has the same 1.5 mm travel, Framework’s keyboard still loses by quite a large margin. Also the keyboard features the rather bad (in my opinion) layout of arrow keys.

This is not to say the built-in keyboard is completely unusable as such. But I do still prefer using a ThinkPad keyboard, whenever it’s available to me.

Customer Service

This is where Framework really shines as a new player in the world of laptop manufacturers. I wouldn’t expect to get in touch directly with someone on their hardware team by simply reporting a firmware issue.

So I decided to try out the latest development version of memtest86+. I built it myself, however it booted on all my machines but the Framework laptop. I reported the issue to memtest86+ developers, and came to the conclusion that it has something to do with the firmware. So I contacted support, and linked the github issue in the support request. The support team wrote back pretty fast (within 24 hours), and to my surprise, agreed to connect me with the hardware team directly. After a few back and forth between the hardware team and memtest86+ developers, it was determined that the cause is 8254 timer gating. Later I heard back from the person on the hardware team that the relevant setting will be added in BIOS version 3.09, which was released a few days ago. I installed the update and set 8254 timer gating to disabled, and memtest86+ worked as expected.

That was my only, yet pretty bizarre experience of Framework’s customer service. I don’t think there could be any chance ever that I could get in touch with someone on the hardware team of any prominent laptop brands, but Framework can do it. Kudos to the Framework support team, and I hope they would be able to maintain this level of customer service when they get bigger.

The full issue report on github is here.

Y no upgrade to 12th gen?

Intel’s 12th gen processors are winning grounds on the desktop front, but in the mobile sector they don’t look so hot. I’ve seen people with their shiny new 10th gen X1 Carbon complaining about poor battery life and even worse thermal performances. Also in order to upgrade without letting the original motherboard fall into disuse, I would either buy new RAM and SSD for it, or sell it somehow. Either option is hard to justify for me. That’s why I decided against upgrading for now. I may consider upgrading to 13th gen whenever that gets released.

Any buyer’s remorse?

Hate to break it for you, but I’m still not daily driving my Framework laptop – in fact, I’m typing this sentence on my old X1.

But there’s no buyer’s remorse in general. Not being a daily driver doesn’t mean it has fallen out of favor. Instead, there are now clear roles assigned to these two laptops. I’ve moved all my music production setup and games to the Framework laptop. It also functions as a build server now (both for binary Gentoo packages, and as a networked build server). The X1 only functions as a typewriter and web browser nowadays.

All current issues I had with this machine have either been resolved, or have existing workarounds. There is no trackpoint, no WWAN, no touch screen or digitizer, or connecting to 3 USB devices and an external monitor while charging compared to my old X1 – this is not a ThinkPad after all. But it does have third party board level repair with support from the company itself (they have started to provide full schematics to third party repair shops under NDA), easy access to parts, open source firmware, upgradable platform, and promise of a coreboot BIOS. These are the reasons I chose to support Framework as a company. So let my first purchase be a small contribution to them.