BAPCo and Futuremark Benchmarks

BAPCo's SYSmark 2014 SE is an application-based benchmark that uses real-world applications to replay usage patterns of business users in the areas of office productivity, media creation and data/financial analysis. In addition, it also addresses the responsiveness aspect which deals with user experience as related to application and file launches, multi-tasking etc. Scores are meant to be compared against a reference desktop (the SYSmark 2014 SE calibration system in the graphs below). While the SYSmark 2014 benchmark used a Haswell-based desktop configuration, the SYSmark 2014 SE makes the move to a Lenovo ThinkCenter M800 (Intel Core i3-6100, 4GB RAM and a 256GB SATA SSD). The calibration system scores 1000 in each of the scenarios. A score of, say, 2000, would imply that the system under test is twice as fast as the reference system.

We see that the patching does have an impact on the performance. It ranges from around 5% for the Office Productivity scenario to around 10% for the Responsiveness metric.

SYSmark 2014 SE - Office Productivity

SYSmark 2014 SE - Media Creation

SYSmark 2014 SE - Data / Financial Analysis

SYSmark 2014 SE - Responsiveness

SYSmark 2014 SE - Overall Score

SYSmark 2014 SE also adds energy measurement to the mix. We see that the patched system, despite being slower, consumes lower energy for the completion of the tasks.

SYSmark 2014 SE - Energy Consumption - Office Productivity

SYSmark 2014 SE - Energy Consumption - Media Creation

SYSmark 2014 SE - Energy Consumption - Data / Financial Analysis

SYSmark 2014 SE - Energy Consumption - Responsiveness

SYSmark 2014 SE - Energy Consumption - Overall Score

Futuremark PCMark 10

UL's PCMark 10 evaluates computing systems for various usage scenarios (generic / essential tasks such as web browsing and starting up applications, productivity tasks such as editing spreadsheets and documents, gaming, and digital content creation). We benchmarked the configurations with the PCMark 10 Extended profile and recorded the scores for various scenarios. These scores are heavily influenced by the CPU and GPU in the system, though the RAM and storage device also play a part. The power plan was set to Balanced for all the PCs while processing the PCMark 10 benchmark.

Similar to SYSmark 2014 SE, we see performance impacts in all the PCMark 10 scenarios also. For example, the Essentials workload score for the fully patched system is around 6.5% lower than the unpatched system. The gaming workload doesn't seem to be affected much by the patching, though. The overall score is around 2.3% lower.

Futuremark PCMark 10 - Essentials

Futuremark PCMark 10 - Productivity

Futuremark PCMark 10 - Gaming

Futuremark PCMark 10 - Digital Content Creation

Futuremark PCMark 10 - Extended

Futuremark PCMark 8

We continue to present PCMark 8 benchmark results (as those have more comparison points) while our PCMark 10 scores database for systems grows in size. PCMark 8 provides various usage scenarios (home, creative and work) and offers ways to benchmark both baseline (CPU-only) as well as OpenCL accelerated (CPU + GPU) performance. We benchmarked select PCs for the OpenCL accelerated performance in all three usage scenarios. Since these scores are heavily influenced by the CPU in the system, it is not surprising to see a performance impact. The patched configuration, on average, scores around 2% lower than the unpatched one.

Futuremark PCMark 8 - Home OpenCL

Futuremark PCMark 8 - Creative OpenCL

Futuremark PCMark 8 - Work OpenCL

Miscellaneous Futuremark Benchmarks

Futuremark PCMark 7 - PCMark Suite Score

PCMark 7's PCMark Suite shows a 4% performance loss.

Futuremark 3DMark 11 - Entry Score

3DMark 11's entry level score and 3DMark 2013's Cloud Gate score actually end up better after the patch, but, 3DMark 2013's Ice Storm score shows a performance loss of more than 13%.

Futuremark 3DMark 2013 - Ice Storm Score

Futuremark 3DMark 2013 - Cloud Gate Score

Futuremark PCMark 8 Storage Bench

PCMark 8 has a storage bench where certain common workloads such as loading games and document processing are replayed on the target drive. Results are presented in two forms, one being a benchmark number and the other, a bandwidth figure. We ran the PCMark 8 storage bench on the ADATA NVMe SSD in all the configurations.

Futuremark PCMark 8 Storage Bench - Score

Futuremark PCMark 8 Storage Bench - Bandwidth

While the absolute score is just around 1.3% lower for the patched configuration, the storage bandwidth is around 29% worse. This significant performance loss is partly due to the NVMe drive performance now being CPU bound. It is highly likely that SATA drives will not experience such a drastic performance loss due to the patching.

Introduction and Evaluation Setup Miscellaneous Benchmarks
Comments Locked

83 Comments

View All Comments

  • akula2 - Sunday, March 25, 2018 - link

    Really, are you stupid or what?

    Don't you know what is happening, Snowden and Wikileaks? You've the audacity to call me as a conspiracy theorist, you moron where were you when Dual_EC_DRBG "vulnerability" was discovered? Who planned that backdoor? Don't you know the collusion of companies behind the epic data collection programme launched by the NSA? Do you really think people on this planet are idiots?
  • boeush - Monday, March 26, 2018 - link

    Yes, Asia - that global bastion of freedom, democracy, open-source transparency, and total absence of corruption. From China, to Vietnam, to North and South Korea, to India and Pakistan, to Myanmar and Indonesia, Afghanistan, Iran, the Philippines, Japan, oh my... LMAO
  • Matthmaroo - Saturday, March 24, 2018 - link

    Someone forgot some meds today
  • TrevorH - Saturday, March 24, 2018 - link

    I would love to see some linux testing added to this set up. I work for a VoIP provider and run CentOS on a bunch of servers and the results I see by adding the initial RH patches were an approximate 30% increase in cpu time. Adding the microcode patch and enabling the IBRS mitigation to that resulted in a 100% increase in cpu usage for our workload. Yes, 100% increase - so a machine with 20 cores that was running at 800% cpu usage before the patches was using 1600% (16 cores at 100% each) after both the PTI and IBRS mitigations were turned on. Now our workload is probably quite unusual in that it uses both KVM virtualization and does lots and lots of small packet UDP network i/o but it does mean that with the mitigations in place, in order to run the same workload that we did before, we'd need to buy just about double the hardware we currently have in use.
  • timecop1818 - Sunday, March 25, 2018 - link

    Hey but why the fuck did you even install the patches, or enable these "fixes"? You've just said it, you are running a closed voip routing system. Why do you need to care about either of these non-problems in those servers? Why do people running Windows on desktop in single user setup need to care about any of this? In an earlier comment Ryan mentioned there's registry settings to disable this, guess what I'm doing as soon as I'm home?
  • Alexvrb - Saturday, March 24, 2018 - link

    You need to test with older architectures. Pre-Broadwell
  • kn0w1 - Sunday, March 25, 2018 - link

    Here is one limited comparison for Ivy Bridge and Y-Series Broadwell for good measure.
    https://www.smajumdar.com/2018/03/musing-48-impact...
  • HStewart - Saturday, March 24, 2018 - link

    Well first all - i not sure if the average customer will even notice these changes.

    Here is two things I thinking of

    1. Has there been actual virus / attack with this stuff - or all of this hypothetical
    2. It odd that power consumptions actually improved with the path

    The good news this stuff is final over with - move on.
  • satai - Sunday, March 25, 2018 - link

    "The good news this stuff is final over with."

    Probably not. We can expect more types of Spectre and similar attacks to come...
  • 29a - Saturday, March 24, 2018 - link

    I have to give props to ASRock for releasing a new BIOS for my Z170 Extreme3 motherboard dated 2018/3/12, I wasn't sure if they would update a budget Z170 board. This along with every ASRock MB I have owned being super stable has made me a very loyal customer.

Log in

Don't have an account? Sign up now