Under the Hood of PCI Express – #4

EEs Talk Tech

Peripheral Component Interconnect Express, officially abbreviated as PCIe® or PCI Express®, is a computer expansion bus standard designed to replace the older bus standards such as PCI.

PCIe 4.0 is doubling the data rate of PCIe 3.0 and poses some interesting challenges for designers. Learn more about PCIe, what it is and how it affects your PC’s performance & capabilities. Daniel Bogdanoff, Mike Hoffman, and Rick Eads discuss.

https://eestalktech.com/wp-content/uploads/2017/04/under-the-hood-of-pcie.mp3

Video (YouTube):

Discussion Overview:

Intro 00:00

Rick Eads is “PCI-Eads” 13:00
Rick’s background with PCIe – He’s been around since the beginning PCIe 1.0
Rick spent time on the PCI-SIG board of directors 50:00
PCI-SIG means “Peripheral Component Interconnect Special Interest Group”
Signal integrity, transmitters, receivers (PCIe PHY Layer) 1:00
What is PCIe? What is PCI Express? 1:20
PCIe means Peripheral Component Interconnect Express
Used to have the ISA bus, ISA means Industry Standard Architecture 1:40
Which transitioned to PCI, but that wasn’t fast enough 1:55
Gaming has driven overclocking 2:00
PCIe is revolutionary 2:25

PCI and ISA was parallel but, PCIe is serial 2:40
PCIe is scalable 3:00
PCIe lanes use lanes that are a differential TX (transmit) and a differential RX (Receiver)3:09
You can have 1 lane (x1, “by one”) up to at least 32 lanes 3:30

PCIe is starting to be used for storage 4:35
Storage with PCIe is popular thanks to solid state drives
NVMe (also called NVM Express) uses PCIe PHY layer 5:25
NVMe means “Non-Volatile Memory Host Controller Interface Specification”
SATA & SAS involve sectoring and writing onto “RUST” (iron oxide) It turns out that sand is faster than rust! 6:00
It’s in consumer-grade equipment now 6:45
m.2 is on most motherboards, and takes an SSD drive running NVMe or SATA express
NVMe is targeted more towards servers but NVMe’s speed and reliability makes it more of a standard interface 7:40

When is PCIe used? 8:05
PCIe is designed to be an interoperability 8:20
The “Root Complex” is the host, like a motherboard, the “Peripheral” is the card 8:40
Why does the PCI-SIG exist? 9:15
The point is that you can be interoperable, the device should work with all other similar devices

PCI-SIG holds interoperability workshop events 11:00
The first rule of the interoperability workshop is we don’t talk about the interoperability workshop 11:30
Engineers have brought in products covered in giant trashbags with a port sticking out 11:50

Thunderbolt is a combination of PCIe and Displayport 13:00
But it doesn’t leverage the “collateral,” the written spec but instead borrows the theory of operation from PCIe
How does Thunderbolt work with the PCI-SIG? 13:30

PCI express is a variant of the PHY portion of InfiniiBand (IB), which is a computer networking standard
A Physical layer is called a PHY
Motherboard designers don’t want to build a lot of PHYs, they want something universal 14:15
Industry tries to build universal PHYs 15:00

PCI express is used in some mobile devices 15:10
Because PCI express has some low power states
Some cars use a PCI express connection for connecting the rear view camera to the rear view mirror display

How far can you transmit PCI express? 16:15
There is a cable version of PCI express
You can use an active cable 16:30
There are some proprietary systems that use a repeater
The repeater transmits a proprietary signal to a receiver that converts it back to PCIe

PCI express 4.0 (PCIe Gen 4) 17:45
The PCI express Gen 4 feature set has been fixed 18:05
It’s not final-final 18:30
PCIe has used a 0.7 version as a sort of a trial run 18:40
PCIe Gen 3 equalization changed from 0.7 to 3.0 and a nonlinear equalizer was implemented 18:45

PCIe Gen 4 is different 19:15
PCI express Gen 4 looks a lot like PCI express Gen 3 19:30

PCI express Gen 4 20:00
The average link length on a motherboard is 10 inches, a server is about 20 inches
This architecture can’t change, so the speed has to increase

Insertion loss in PCIe Gen 4 is increased 10:50
As loss increases, the ability to transmit data decreases
PCI express Gen 3 runs at 8 Gb/s per data link, PCI express Gen 4 runs at 16 Gb/s per data link
PCI express Gen 3 has a 25 mV eye height 21:00, PCI express Gen 4 has a 15 mV eye height
and even then, the lane length is too long 21:30

PCI express Gen 4 adds a retimer (a form of a repeater) 22:05
A retimer was option for PCI express Gen 3, but it’s refined in Gen 4
A retimer takes data and passes it through as quickly as possible to extend the channel length 22:35
But, it takes more power and more cost. Designers are not excited about retimers 22:50

PCI express Gen 4 also uses a lower loss material 23:00
PCI express Gen 4 uses FR4 23:10
FR4 stands for “fire retardant 4”
Electronics fail because the magic smoke comes out 23:45
FR4 is cheap, which is why engineers are using it 23:59
The motherboard cost can double just by going to a lower loss interconnect material 24:10
FR4 is a weaved fiberglass weave put in a mesh and pressed
You can have variations in insertion loss characteristics, etc 25:05

Predictions 28:10
High speed PCI express buses will appear in weird places like IoT toasters!

Bonus 32:19