1 of 20 August 18, 2009
© 2009 Integrated Device Technology, Inc.
Notes
®
IDT Confidential
Introduction
With the increases in PCIe® bus speeds and system performance requirements, the ability to utilize the
PCIe bus bandwidth without adding additional load to the host CPU becomes increasingly more important.
A simple PCIe GigE NIC solves this problem by integrating a simple DMA engine, allowing for maximum
data throughput with minimum host CPU overhead. Of course, this DMA engine is dedicated to the NIC,
which makes it useless when it comes to transferring data to or from other endpoints in the system. As
PCIe-based system topologies become more popular, there is a growing need for a generic method to
move large amounts of data quickly between the root complex and the endpoints, or between multiple
endpoints. The solution to this problem is to integrate flexible DMA engines directly into the PCIe switch.
DMA Overview
The IDT PES32NT24G2 PCIe switch contains two high performance DMA engines, each supporting 2
independent channels. A DMA engine can serve as a Function of the partition’s upstream port (shown in
Figure 1), as a Function of a Non-transparent port (shown in Figure 2), or as a Function of an Upstream
Port with a Non-transparent Port (shown in Figure 3).
Figure 1 Upstream Port with DMA Function
P2P
Bridge
DMA
Function
Virtual PCIe Bus
P2P
Bridge
Downstream Ports
P2P
Bridge
P2P
Bridge
P2P
Bridge
Upstrea m Port
By Craig Hackney
Application Note
AN-714
DMA in PCIe® Switches
2 of 20 August 18, 2009
IDT Application Note AN-714
Notes
IDT Confidential
Figure 2 Non-transparent Port With DMA Function
Figure 3 Upstream Port with Non-transparent Port and DMA Function
The DMA channels perform fly-by translation of the TLP’s to reduce latency and increase performance
over buffered approaches.
Figure 4 illustrates a data transfer operation performed by a DMA channel. As
shown in this figure, the DMA channel issues memory read request TLP’s (MRd) to read data from the
source memory (1). When the DMA receives a completion (Cpl) TLP for the read request (2) it transforms
the completion into a memory write (MWr) TLP and issues it to the destination memory (3).
P2P
Bridge
DMA
Function
Virtual PCIe Bus
P2P
Bridge
Downstream Ports
P2P
Bridge
P2P
Bridge
NTB
Upstrea m Port
Non-transparent
Port
NTB
NT Interconnect
P2P
Bridge
DMA
Function
Virtual PCIe Bus
P2P
Bridge
Downstream Ports
P2P
Bridge
P2P
Bridge
NTB
Upstrea m Port
P2P
Bridge
To other
pa rtitio n