NUCserver Hardware Development Project Status

 

  1. Proprietary IP-PDU – Completed and Launched

 

We first created an IP-addressable power control and distribution device for sets of 8 NUC’s. The IP-PDU allows for remote power management of individual ports, and we applied it to the NUCserver project. In order to adapt it for a conventional server rack environment, we downsized the power brick. Although it was designed for enterprise-level embedded environments, the fact that it is PC-based was creating some anxiety among consumers over its reliability. Creating this device was an inevitable decision we had to face in order to meet the needs of our clientele.

 

Figure 1. NUCserver Universal Solution

 

  1. Proprietary miniDP to Headless Dongle – Completed and Launched

     

    Servers that support IPMI, a widely known remote management technology, often include an analogue graphics chipset that enables features like remote power control and KVM remote desktop. In contrast, Intel’s vPro technology used for NUCserver was developed for purposes such as large-scale enterprise computing, industrial computing, digital signage and ATM machines. So it was developed for PC and workstation management, without much consideration for environments without monitors. The NUC5i5MYHE we picked for the NUCserver, in particular, had two Mini DisplayPorts, but had a drawback where remote KVM would not work without a monitor attached. In order to overcome this limitation, the industry launched many types of Headless Plug products which are easily purchased through websites like eBay. NewerTech had the smallest Mini DisplayPort to Headless dongle, called the Headless Mac Video Accelerator, created for Mac Mini.


    Figure 2. NewerTech Headless Mac Video Accelerator

     

    The problem was that this product is more expensive than it sounds, and as seen in Figure 1, it would not fit in the tightly-packed NUCserver universal solution containing 8 NUC’s. We decided to create our own. It took us multiple trials and errors. We had to look at the circuitry of a Chinese adaptor and in the end, it had taken us over four months to step through the prototypes in Figure 4 and 5, and finally reaching the final model in Figure 6. This dongle enables the use of remote KVM without a monitor.


    Figure 3. Chinese MiniDP to RGB Adaptor Used for Reference


    Figure 4. Prototype 1            Figure 5. Prototype 2          Figure 6. Completed Dongle

     

  2. Second Network Interface Development Project Status

     

    With the development of the IP-PDU and MiniDP to Headless Dongle, the NUCserver project was in full throttle. The initial NUCserver launch period in late 2015 coincided with a worldwide boom in cloud computing. Among the cloud platforms were OpenStack, which quickly became influential. We realized in our attempt to adapt our independent NUCservers into a cloud that in order to use OpenStack’s compute node, Nutron, we will need at least 2 network interfaces per device. We could have opted for using VLAN with the existing port to sidestep the limitation of having a single Ethernet port, but it would degrade performance. In order to avoid such degradation, we searched for a module that would fit the NUC5i5MYHE, which are the NUC’s used for NUCservers. Fortunately, we found a dual-port NGFF network interface sold by Innodisk, a Taiwanese manufacturer. Despite the relatively high price, we happily purchased a sample.


    Figure 7. Innodisk’s NGFF Dual Network Card Kit

    The trial went terribly. The interface was not recognized by the NUC, bridging through USB 3.0 caused overheating, and the thick cables made it clear that we could not add this module to the NUC. To circumvent this issue, we ended up developing our own circuitry, working with local ODM’s in China and Taiwan to avoid the complicated process of production.

        

    Figure 8. Design            Figure 9. Proprietary RJ45 Connector Size Limit

    There were two main design challenges. The first was minimizing the size to make it fit in the case, and the second was to find a RJ45 port that fits the opening in the case, which is relatively small at only 11 mm in height. Luckily, we were able to find a Taiwanese manufacturer selling Ultra Low Profile connectors used in Ultrabooks, and we proceeded in collaboration with a Chinese local ODM.

           

    Figure 10. Prototype 1            Figure 11. Prototype 1 Installed          

    As shown in Figure 11 and 12, we connected our first prototype, but we couldn’t fit it in. It was working correctly, however.

       

    Figure 12. Prototype 2 Installed            Figure 13. Prototype 3 Installed   

    Prototype 2 shown in Figure 12 was difficult to connect and integrate despite the longer module and shorter cables, because the distance between the connectors was too short. In Figure 13, we soldered the cable onto the daughter board in order to shorten the cable, which took care of reliability and connectability, but it was likely to lose contact after multiple uses.


    Figure 14. Completed Production-ready Model Exterior View

    We found that the production-ready model had increased reliability and compatibility. Performance tests also showed satisfactory results.


    Figure 15. Production-ready Model Performance and Installation Test

    Completed through a long development process, the True Networks Second Network Interface for NUCservers will finally be launched worldwide as early as May, after passing Intel’s certification process as requested by True Networks, an official Intel solutions partner.

Advertisements
기본

What Makes the NUCserver a Server?

The NUCserver project, which uses Intel’s NUC Mini PC’s as servers, began with a simple question: what is a server?

041116_1805_17.png

True Networks has been a NSP (Network Service Provider) for more than 5 years, providing services and rentals through the servers installed in our data centers. Like most hosting businesses, the largest portion of our operating costs go towards physical space and leased lines.

 

In case of our Japanese branch which called True Networks Japan which provides the same services as True networks, their operating cost is at least 4 time higher then Korea. We have pored over and applied various way to cost-cutting to become more profitable, but we have always hit the physical barrier.

 

Let’s return to the initial question: “what is a server?” According to Wikipedia, a server “is a system or program that responds to requests across a computer network to provide, or help to provide, a network or data service. According to this definition, a server, as a hardware, is a computer device that serves a specific function or a role from a remote location through a network, physically inaccessible unlike a PC or a workstation. As a server needs to perform continuously for long periods of time in an environment where the user or an administrator cannot manage it directly, reliability and remote manageability—biggest differentiators separating servers from PC’s and workstations—are of utmost importance to a server.

 

Of course, reliability is highly desirable for any PC or business PC, and high-performance enterprise workstations for jobs like graphics rendering, architecture and statistics are expected to have server-level reliability. However, it is not a requirement, and the need for remote management is relatively low.

 

A server in the mind of an average Joe is intimidating machinery, much more powerful and scalable than the average PC, but quite to the contrary, a steady stream of recent articles has been describing the use of devices built on low-power processors, like the Raspberry Pi, performing as servers with limited function.

By all means, large servers have an edge in computing capability, necessitated by tasks like cluster supercomputing, rendering graphics and statistics, since its processing power, memory, storage, and I/O can be expanded to the maximum. Nevertheless, the performance necessary for web, file, DNS or email servers for personal use or small businesses is relatively low, and the same goes for servers with limited roles.

 

The ideal server hardware therefore needs:

  1. Performance for handling the services to be provided to users over the network
  2. Reliability for continuous performance during the service term
  3. Remote manageability for emergencies, troubleshooting and maintenance

 

To get a little more greedy, we could add great price-performance ratio, low maintenance costs, and good technical support to the list. But expandability and high performance are still not requirements.

They are merely specifications that depend on the purpose of the server.

 

The NUCserver project began with this reexamination of what defines a server.

We here at True Networks have learned a thing or two in the past years:

  1. The average load of servers we operate or manage never goes above 10%. The fixation on performance is creating an excess.
  2. Customers demand high-performance servers with Scalability in mind, but less than 10% of our customers actually have needed upgrades due to expansions. In addition, the majority of customers who upgrade buy an additional server instead of adding storage or computing power.
  3. Although the demand for public cloud for services like virtual machines have been rising, customers often have second thoughts about its security and reliability in comparison to a private server. In these cases, they end up creating a private cloud, which often takes a lot more resources than expected.
  4. Even though physical, independent servers will no longer be mainstream and are in decline for high costs, many customers want to rent one or have one installed in our data centers.
  5. A decade of price war has wreaked havoc on the profit margin of most players in the virtual data center industry, and as they reach the conclusion that competing on service is essential, companies that have the luxury of affording R&D for new services and management systems are the few that were able to maintain their revenues. Cost cutting remains vital.
  6. In order to become more cost competitive, we need server systems with high space efficiency and low power consumption that will allow us to cut operating costs.

 

These are widely agreed upon, but let’s focus on number six.

As a solution to the power and space efficiency problem, many server manufacturer, including Dell, HP and Super Micro, have offered high-density blade servers, proving effective in many aspects. Many large-scale enterprises have adopted blade servers. Nonetheless, blade servers still have unsolved problems.

 

  1. Many enterprises adopt blade servers with the idea that the total cost of ownership (TCO) will be lower since the cost of adoption is similar to conventional servers while being more efficient in terms of power and space. This may be true, but maintenance costs are higher than conventional servers because proprietary components and specialized engineers for proprietary hardware prove to be expensive.
  2. Focusing on power efficiency and performance per unit of space has led to high computing power in each blade. This makes it difficult to respond to the demand for a private, moderate-level performance servers.
  3. The maximum power supply per unit space, set by the data center operator, is relatively low, which causes some problems when adopting blade servers. For example, when designing a general-purpose data center in Korea or Japan, the power supply limit per standard 42U rack is 220 V at 30 A, which is 6.6 kW (110 V at 60 A in Japan), but for safety, this is usually held back at around 3 kW. For more power supply, customers must individually enter a contract with higher fees. With this limit, considering that the average 4-6U 12-slot blade servers require between 1 and 2 kW, 12-20U, or three set of blade server is the maximum installation on a single rack. This leaves about half of a standard 42U rack empty, which can only be filled with additional power with additional fees (usually 1.5 times the normal fee). Ultimately, its space-power-cost efficiency is low.

 

These observations have led to the specifications for our server design, the solution to the aforementioned problems.

 

  1. Appropriate performance, enough to provide the services required by our clients.
  2. Reliability nearing or identical to conventional servers.
  3. Manageability nearing or identical to conventional servers.
  4. Staying within the power limit of 6 kW per rack while providing enough performance per module for an independent server. Achieving low TCO even after the additional fees by maximizing per-module power and space efficiency.
  5. Low maintenance costs by using standard components.

 

When Intel first revealed the NUC in 2012, we did contemplate using it as a server, only to abandon it shortly after. The primary reason was that IPMI-like functionality for remote power control and management was essential for a server. At the time, we were searching for a PicoITX-sized E3-1200 processor based product that had integrated IPMI, and we were even ready to become an ODM, design our own, or collaborate with existing server board manufacturers like Iwill and Tyan Korea.

 

DC53427HYE, a new NUC model released by Intel in Q4, 2013 included the vPro technology, which provided a robust management system that rivaled IPMI, with better security. It caught our attention. This particular model had the Sandy Bridge i5 processor, which was new at the time, and unlike other NUC devices, it was designed as an embedded device, providing a very long MTBF and reliability.

 

When we had our hands full trying to acquire a sample and implement our design, the market in 2014 was overflowing with Ivy Bridge models. But the model with the new processor and vPro technology was nowhere to be seen. It was hugely disappointing when we made an enquiry about the vPro model and learned that they were uncertain about future releases. We dropped the project soon after.

 

This model still had some flaws. The small size made it incompatible with standard 2.5” SSD’s and HDD’s. It came with 16 GB of memory, which made is suitable for some cases, but the capacity of mSATA SSD’s maxed out at a meager 256 GB.

 

In early 2015, Intel released a sequel to the NUC model with vPro, called the 5i5MYHE. This model was a step closer to the server we had in mind. Full of anticipation, we acquired a sample, and as a result, we rebooted the NUCserver project. We developed a standard and a proprietary rack system, applications and a power control device. In order to overcome a hardware limitation that doesn’t allow for remote desktop access, we also created a MiniDP to headless (a virtual monitor emulator) dongle.

 

Let me take you through what we’ve achieved in the past year.

  1. Reliability nearing or identical to conventional servers

→ Embedded design (MTBF 62,000 hours)

  1. Manageability nearing or identical to conventional servers

→ Development of a power/installation/management control system based on the vPro technology.

→ Development of the MiniDP to headless dongle

  1. Staying within the power limit of 6 kW per rack while providing enough performance per module for an independent server. Achieving low TCO even after the additional fees by maximizing per-module power and space efficiency.

→ Developing a rack system for equipping a standard 42U rack with 192 NUC’s and an IP-PDU with a power supply device, maximizing performance per watt while keeping below a 5 kW total.

→ Moderate performance using the Broadwell i5 processor and the latest HD Graphics technology.

  1. Low maintenance costs by using standard components.

→ Keeping the hardware standard using the NUC5i5MYHE kit

  1. Expandability (added)

→ Ability to install a 2.5” disk and dual NGFF modules (standard)

→ Ability to install a maximum of 32 GB of ECC memory (option)

→ Ability to install an extra gigabit Ethernet port (option)

Once the development of a NGFF-based (M.2) gigabit network interface is completed in Q1 2016 along with a proprietary daughter board and NUCserver brackets, the NUCserver will be nothing short of a true server.

041116_1807_17.png

기본

누크서버 프로젝트를 위한 하드웨어 개발 프로젝트 현황

  1. 전용 IP-PDU – 완료 출시

먼저 8대의 NUC 세트로 구성하여 전용 전원공급장치를 겸하여 포트별 물리적 전원 제어가 가능 한 IP베이스의 원격전원 제어 공급장치를 개발하여 NUCserver 프로젝트에  적용하였다. 표준 42U 랙을 사용하는 기존 서버와 동일한 환경에 적용하기 위해 전원부의 부피를 줄이고 관리 편의성을 높이면서 동시에 산업용 임베디드 환경에 맞게 설계되었음에도 불구하고 PC기반이라는식의 한계로 인해
기존 서버  대비 안정성을 염려하는  고객들의 불안감을 불식시키기 위한 불가피한 선택이었다.


그림1. 누크서버 범용 솔루션

  1. 전용 miniDP to Headless Dongle – 완료 출시

서버의 원격 관리용으로 널리 알려진 IPMI 방식을 지원하는 대부분의 서버들의 경우 아날로그 그래픽 칩셋을 내장하여 전원제어는 물론 KVM 원격 화면제어 기능을 구현하고 있다. 이에 비해 NUCserver 채용한 인텔 vPro 테크놀로지는 대규모 기업용/ 산업용/Digital signage/ATM 등을 염두에 두고 개발한 워크스테이션 PC 관리툴로 개발되어  일반적으로 모니터가 없는 환경을 고려하지 않았다. 특히 당사가 NUCserver용으로 채택한 NUC5i5MYHE 모델은 두개의 Mini Display Port 지원하여 실제 적용시 모니터를 연결하지 않는 경우, 원격 KVM 기능이 동작하지 않는 단점이 있다. 업계에서는 이런 한계를 극복하기 위해 HDMI, miniHDMI, DVI, DP, miniDP, RGB 등의 다양한 디스플레이 포트들을 지원하는 Headless Plug 제품들을 출시했으며 이베이 등을 통해 쉽게구매할 있다. 현존하는 가장 작은 크기의 Mini Display 포트용 Headless plug Newer tech 사가 Mac Mini 제품을 위해 출시한 Headless Mac Video Accelerator 제품이 유일했다.


그림2. Newer tech Headless Mac Video Accelerator

문제는 이 제품의 가격이 생각보다 비싼데다, 그림1누크서버 범용솔루션 예시에서 보이는 것처럼 한정된 공간에 8개나 장착하기 용이하지 못해 결국 직접 개발하게 되었다. 그림 3 의 중국산 어뎁터 회로를 참조하는 등 총 4개월의 개발 기간과 여러번의 설계 실수를 거친 그림4 와 5 같은 시제품 제작과정 후 결국 그림6 과 같은 완성판을 만들게 되었다. 이로서 한정된 공간 제약에도 불구하고 모니터 없이 원격 KVM 기능을 사용할수 있게 되었다.

그림3. 설계 참조를 위한 중국산 MiniDP to RGB Adaptor


 그림4. Prototype1               그림5. Prototype2               그림6. 완성판 동글

3. Second Network Interface 개발 프로젝트 현황.

위와같은 전용 IP-PDU 전용 Mini DP to Headless Dongle 제작에 힘입어 인텔사의 미니 PC NUC
서버로 사용하기 위한 NUCserver 프로젝트는 비교적 순조로이 진행되었다.

이후 초기 NUCserver모델을 시장에 출시하던 시기인 2015 후반기부터 전세계적으로 클라우드
구축 열풍이 거세지기 시작했다. 그중에서도 OpenStack 이라는 걸출한 클라우드용 플렛폼의
영향력이 커지면서 NUCserver 단독 서버가 아닌 클라우드에 적용하기 위한 시도를 이어가던
, OpenStack Compute Node Nutron으로 구성하기 위해서는 최소 2개의 Network Interface
필요함을 인지하게 되었다. 물론 기존의 내장형 Network Interface 이용하는 경우 VLAN 구성하여
적용할 있으나 성능 저하는 피할 없었다.

이를 극복하기 위해 NUCserver 채용한 NUC5i5MYHE 모델에 적용가능한 제품을 검색해본 결과 다행히 대만의 Innodisk 사에서 NGFF 듀얼포트 Interface 출시했음을 확인하고상보다 높은 가격에도
불구하고 기쁜 마음으로 샘플을 수입했다.


그림7. Innodisk 사의 NGFF 규격 듀얼 네트워크 카드 세트

실제 적용 결과는 처참했다. 인터페이스는 아예 NUC 에서 인식조차 되지 않았고, USB3.0브릿지를 거치면서 발생하는 상당한 발열문제와 두꺼운 연결 케이블로 인해 내장이 불가함만 확인하게 되었다. 결국 자체 개발로 선회하였으나 실제적인 조립/생산 등의 복잡한 절차를 피하기 위해 중국/대만 등의 현지 공장을 통한 ODM으로 진행하게 되었다.


그림8 기본설계 그림           9. 전용 RJ45 콘넥터 크기제한

설계상의 문제는 크게 두가지 였다 크기를 최소화 하여 케이스에 내장하는것과 외부 홀의 두께가 11mm 로 낮아 그에 맞는 RJ45 콘넥터를 구하는 것이였다. 다행히 콘넥터는 최근 유행하는 울트라북에 사용하는 Ultra Low Profile 제품을 대만의 한 제조사에서 구입할 수 있었고 이후 선정한 중국 현지 ODM 파트너의 도움으로 개발을 진행했다.


그림10. 프로토타입 1                                           그림11. 프로토타입1 적용

그림 10 11 처럼 프로토타입을 적용한 결과 케이블이 너무 두껍고 길어서 실제 내장에 실패했으나 기능상 이상은
발견하지 못했다.

그림12. 프로토타입 2 적용                                                   그림13 프로토타입 3 적용

그림 12 프로토 타입 2에서는 카드의 길이를 늘리고 케이블 길이를 짧게 했음에도 불구하고 콘넥터간 간격이 너무 짧아 컨넥터채결과 내장이 어려웠다. 그림 13 경우 케이블 길이를 줄이기 위해 Daughter board 케이블을 soldering 하여 안정성과 체결성을 확보했으나 여러 접촉할 경우 케이블 단선이 우려되는 상황이었다.

그림14 생산 준비 모델 완성과 실제 적용 외관

완성된 생산 준비 모델을 적용한 결과 체결성과 안정성이 높아지고 실제 성능 테스트 결과도 만족스러운 수준이다.


그림15 생산 준비 모델 성능 및 설치 테스트 결과.

이렇게 지루한 제작 과정을 거쳐 완성된 NUCserver 전용 Second Network Interface 는 Intel 사의 공식 솔루션 파트너인 트루네트웍스의 공식적인 요청에 의해 Intel 사의 내부 검증을 거쳐 이르면 5월 중 전세계적으로 출시될 예정이다.

기본