Reporting CPU Usage on CDN Pro

Why

Today's content delivery networks (CDN) are doing much more than just caching and bit pushing. All the major players are trying to extend their business into the territory of edge computing. Advanced platforms, such as CDNetworks' CDN Pro, offer programmability to support sophisticated logic on the edge servers. Some logic, such as that for cryptographic algorithms, regex manipulations and data compression, can be extremely computational intensive. Running them on the CDN edge not only offloads these resource-intensive activities from the origin, but also reduces latency significantly to provide end users an exceptional experience.

However, this trend is not yet reflected in the price model in the industry. Up to today, most CDN providers still charge primarily based on the traffic volume (bytes delivered) or bandwidth (mostly 95p). This model was created decades ago when the majority of CDN costs were for the bandwidth procured from the ISPs. It worked well when CDN was used only to deliver highly cacheable large files (HCLF); however,  this is no longer the case with today's scenarios. For example, a modern server can easily deliver 40Gbps of HCLF; but to accelerate a dynamic API service of the same bandwidth across the Pacific Ocean, 500 servers may be required. The data center cost is largely proportional to the number of servers. This huge disparity in resource consumption by different services created many challenges for both the CDN providers and their customers about pricing.

To ensure profitability, CDN providers have to merge all extra costs into a per-GB or per-Mbps price. But before a service is completely onboard, it's not feasible to accurately estimate the amount of extra cost it's going to generate. Customers can usually provide a ballpark number for the traffic volume or bandwidth, but they can't tell how many servers will be needed. Not to mention that they may change their business logic at any time through the self-service interface. It takes a lot of guesswork to set the price based on the traditional model. As a result, it is nearly impossible to create a price book that is fair for all customers and service types. The solution is actually quite simple: introducing a charge for CPU consumption. CPU has been a billing item of cloud computing since its inception. It should come as no surprise, therefore, to see it adopted by edge computing, which is what CDN is evolving into.

How

Since the resources of a CDN platform are shared across multiple customers, it is not easy to measure the usage by each individual one. Even for traffic volume, the most commonly used billing item, only data from the application layer can be accounted for accurately. It is quite tricky to collect the network overhead at lower layers and attribute it to each customer, so most providers decide not to bother with this. The measurement of CPU utilizations of each customer can be even more challenging, however, we identified this as a must-have feature at the beginning of the CDN Pro project back in 2016. When we decided to build the edge server based on NGINX, our team started to make extensive changes to the open-source version.

NGINX has an "event-driven, non-blocking" architecture. Every request is put to sleep when waiting for I/O, awakened when some data is ready to be processed, and then put to sleep again when the processing is completed. This can occur many times during the lifecycle of a request. We use the function clock_gettime() on Linux to obtain the amount of CPU time, in nanoseconds, spent on each active interval of each request and accumulate along the way. At the end of the request's lifecycle, we write the total consumed CPU time into the access log just like how we treat the bytes transferred. The CPU time is also included in the traffic summary log to generate the low-latency, per-domain reports.

While this method can accurately measure the CPU time consumed by the NGINX processes on each request, it does not include the following:

• Kernel processing the interrupts from the network interface cards or other I/O devices.
• Common tasks performed by the NGINX main process.
• Other services running on the same server, such as log preprocessing, monitor, and so on.

Data

As mentioned above, the CDN Pro platform offers a time series report to show the CPU time consumed by each domain. It returns this metric, in seconds, in each time interval of the specified granularity. For more information, refer to the API documentation. This report is similar to the traffic volume report, which returns the number of bytes delivered in each interval; in this case, we can divide the values by the width of the interval to get traffic bandwidth. For the CPU report, the division actually results in the number of CPU cores consumed in each interval. Another interesting metric is the average CPU time per request. This value varies between a few microseconds and a few seconds across different domains served by CDN Pro. According to our observation, here are some of the main factors that affect this value:

TLS handshake: The charts below show the average CPU time per request grouped by the order in the connection. The curves with label "1" correspond to the first request in each connection. The difference between it and the subsequent requests reflects the CPU time consumed by the TLS handshake. We can see that 2ms is consumed on average for an RSA-2048 certificate. ECDSA-256 uses about 0.9ms.  

TLS reuse factor: If a TLS connection is reused by multiple requests, the cost of the handshake is also shared by all those requests.

Cache hit status: A hit usually takes less CPU time. Here is a chart showing this fact:

Size of the response: Bigger responses take more CPU power to deliver:

The last factor above prompted us to think about the following: What CPU power is needed to deliver a fixed amount (for example, 1GB) of data? This can be easily obtained by dividing the CPU time by the traffic volume. As expected, we observed a huge difference across different domains. For domains with only HCLF, this number can be lower than 1 second per GB. But for some highly dynamic API services, this number can be hundreds of times higher.

Billing

The data above clearly illustrates the point made in the first section. It is not fair to charge the CDN service only by the network traffic. It's like charging everything in a supermarket only by their weights. Introducing the billing item by CPU time ensures each customer is charged equitably. It also ensures the CDN service provider a more stable profit margin while opening more edge computing capabilities through self-service. In a few weeks, CDN Pro will officially introduce this new billing item. As a result, the charge for HTTPS requests will be removed because the associated cost is more accurately covered by the new item. At the same time, the traffic volume prices for all server groups will be reduced significantly. Please stay tuned!