networking/dsa/dsa.rst

5 This document describes the **Distributed Switch Architecture (DSA)** subsystem
13 The Distributed Switch Architecture is a subsystem which was primarily designed
14 to support Marvell Ethernet switches (MV88E6xxx, a.k.a Linkstreet product line)
19 they configured/queried a switch port network device or a regular network
22 An Ethernet switch is typically comprised of multiple front-panel ports, and one
24 presence of a management port connected to an Ethernet controller capable of
25 receiving Ethernet frames from the switch. This is a very common setup for all
26 kinds of Ethernet switches found in Small Home and Office products: routers,
27 gateways, or even top-of-the rack switches. This host Ethernet controller will
32 using upstream and downstream Ethernet links between switches. These specific
34 of multiple switches connected to each other is called a "switch tree".
36 For each front-panel port, DSA will create specialized network devices which are
37 used as controlling and data-flowing endpoints for use by the Linux networking
41 The ideal case for using DSA is when an Ethernet switch supports a "switch tag"
42 which is a hardware feature making the switch insert a specific tag for each
43 Ethernet frames it received to/from specific ports to help the management
46 - what port is this frame coming from
47 - what was the reason why this frame got forwarded
48 - how to send CPU originated traffic to specific ports
52 on Port-based VLAN IDs).
57 - the "cpu" port is the Ethernet switch facing side of the management
61 - the "dsa" port(s) are just conduits between two or more switches, and as such
63   downstream, or the top-most upstream interface makes sense with that model
65 Switch tagging protocols
66 ------------------------
68 DSA currently supports 5 different tagging protocols, and a tag-less mode as
71 - ``net/dsa/tag_trailer.c``: Marvell's 4 trailer tag mode (legacy)
72 - ``net/dsa/tag_dsa.c``: Marvell's original DSA tag
73 - ``net/dsa/tag_edsa.c``: Marvell's enhanced DSA tag
74 - ``net/dsa/tag_brcm.c``: Broadcom's 4 bytes tag
75 - ``net/dsa/tag_qca.c``: Qualcomm's 2 bytes tag
80 - identifies which port the Ethernet frame came from/should be sent to
81 - provides a reason why this frame was forwarded to the management interface
84 ----------------------
87 the CPU/management Ethernet interface. Such a driver might occasionally need to
93 Ethernet switch.
96 ----------------------
99 networking stack is in order to have the DSA subsystem process the Ethernet
100 switch specific tagging protocol. DSA accomplishes this by registering a
101 specific (and fake) Ethernet type (later becoming ``skb->protocol``) with the
103 Ethernet Frame receive sequence looks like this:
109         - receive function is invoked
110         - basic packet processing is done: getting length, status etc.
111         - packet is prepared to be processed by the Ethernet layer by calling
114 2. net/ethernet/eth.c::
117                   if (dev->dsa_ptr != NULL)
118                           -> skb->protocol = ETH_P_XDSA
120 3. drivers/net/ethernet/\*::
123                   -> iterate over registered packet_type
124                           -> invoke handler for ETH_P_XDSA, calls dsa_switch_rcv()
128           -> dsa_switch_rcv()
129                   -> invoke switch tag specific protocol handler in 'net/dsa/tag_*.c'
133         - inspect and strip switch tag protocol to determine originating port
134         - locate per-port network device
135         - invoke ``eth_type_trans()`` with the DSA slave network device
136         - invoked ``netif_receive_skb()``
138 Past this point, the DSA slave network devices get delivered regular Ethernet
142 ---------------------
146 controlling and data-flowing end-point for each front-panel port of the switch.
149 - insert/remove the switch tag protocol (if it exists) when sending traffic
150   to/from specific switch ports
151 - query the switch for ethtool operations: statistics, link state,
152   Wake-on-LAN, register dumps...
153 - external/internal PHY management: link, auto-negotiation etc.
157 stack/ethtool, and the switch driver implementation.
160 switch tagging protocol is currently registered with these network devices, and
162 switch tag in the Ethernet frames.
165 ``ndo_start_xmit()`` function, since they contain the appropriate switch tag, the
166 Ethernet switch will be able to process these incoming frames from the
167 management interface and delivers these frames to the physical switch port.
170 ------------------------
176                 |---------------------------
178                 ----------------------------
179                 | <tag added by switch     |
183         |--------------------------------------------|
184         |            Switch driver                   |
185         |--------------------------------------------|
187               |-------|  |-------|  |-------|
189               |-------|  |-------|  |-------|
194 --------------
196 In order to be able to read to/from a switch PHY built into it, DSA creates a
197 slave MDIO bus which allows a specific switch driver to divert and intercept
198 MDIO reads/writes towards specific PHY addresses. In most MDIO-connected
200 to return standard MII registers from the switch builtin PHYs, allowing the PHY
201 library and/or to return link status, link partner pages, auto-negotiation
204 For Ethernet switches which have both external and internal MDIO busses, the
206 internal or external MDIO devices this switch might be connected to: internal
210 ---------------
215 - ``dsa_chip_data``: platform data configuration for a given switch device,
216   this structure describes a switch device's parent device, its address, as
220 - ``dsa_platform_data``: platform device configuration data which can reference
222   the master network device this switch tree is attached to needs to be
225 - ``dsa_switch_tree``: structure assigned to the master network device under
227   the tagging protocol supported by the switch tree, and which receive/transmit
229   switch is also provided: CPU port. Finally, a collection of dsa_switch are
232 - ``dsa_switch``: structure describing a switch device in the tree, referencing
236 - ``dsa_switch_ops``: structure referencing function pointers, see below for a
243 -----------------------------------------
246 (``DSA_MAX_SWITCHES``), and the number of ports per switch to 12 (``DSA_MAX_PORTS``).
251 -------------------------------
256 - inability to fetch switch CPU port statistics counters using ethtool, which
257   can make it harder to debug MDIO switch connected using xMII interfaces
259 - inability to configure the CPU port link parameters based on the Ethernet
262 - inability to configure specific VLAN IDs / trunking VLANs between switches
266 --------------------------------
268 Once a master network device is configured to use DSA (dev->dsa_ptr becomes
269 non-NULL), and the switch behind it expects a tagging protocol, this network
272 will not make us go through the switch tagging protocol transmit function, so
273 the Ethernet switch on the other end, expecting a tag will typically drop this
285 - MDIO/PHY library: ``drivers/net/phy/phy.c``, ``mdio_bus.c``
286 - Switchdev:``net/switchdev/*``
287 - Device Tree for various of_* functions
290 ----------------
296 - internal PHY devices, built into the Ethernet switch hardware
297 - external PHY devices, connected via an internal or external MDIO bus
298 - internal PHY devices, connected via an internal MDIO bus
299 - special, non-autonegotiated or non MDIO-managed PHY devices: SFPs, MoCA; a.k.a
305 - if Device Tree is used, the PHY device is looked up using the standard
306   "phy-handle" property, if found, this PHY device is created and registered
309 - if Device Tree is used, and the PHY device is "fixed", that is, conforms to
310   the definition of a non-MDIO managed PHY as defined in
311   ``Documentation/devicetree/bindings/net/fixed-link.txt``, the PHY is registered
314 - finally, if the PHY is built into the switch, as is very common with
315   standalone switch packages, the PHY is probed using the slave MII bus created
320 ---------
324 of per-port slave network devices. Since DSA primarily deals with
325 MDIO-connected switches, although not exclusively, SWITCHDEV's
327 checks whether the operation is supported by the DSA switch driver, and a commit
334 -----------
339 per-port PHY specific details: interface connection, MDIO bus location etc..
344 DSA switch drivers need to implement a dsa_switch_ops structure which will
353 Switch configuration
354 --------------------
356 - ``tag_protocol``: this is to indicate what kind of tagging protocol is supported,
359 - ``probe``: probe routine which will be invoked by the DSA platform device upon
360   registration to test for the presence/absence of a switch device. For MDIO
362   the switch pseudo-PHY and return whether this is a supported device. For other
363   buses, return a non-NULL string
365 - ``setup``: setup function for the switch, this function is responsible for setting
368   configure the switch to separate all network interfaces from each other, that
369   is, they should be isolated by the switch hardware itself, typically by creating
370   a Port-based VLAN ID for each port and allowing only the CPU port and the
372   platform should be disabled. Past this function, the switch is expected to be
374   to issue a software reset of the switch during this setup function in order to
379 -------------------------------
381 - ``get_phy_flags``: Some switches are interfaced to various kinds of Ethernet PHYs,
383   on its own (e.g.: coming from switch memory mapped registers), this function
384   should return a 32-bits bitmask of "flags", that is private between the switch
385   driver and the Ethernet PHY driver in ``drivers/net/phy/\*``.
387 - ``phy_read``: Function invoked by the DSA slave MDIO bus when attempting to read
388   the switch port MDIO registers. If unavailable, return 0xffff for each read.
389   For builtin switch Ethernet PHYs, this function should allow reading the link
390   status, auto-negotiation results, link partner pages etc..
392 - ``phy_write``: Function invoked by the DSA slave MDIO bus when attempting to write
393   to the switch port MDIO registers. If unavailable return a negative error
396 - ``adjust_link``: Function invoked by the PHY library when a slave network device
398   configuring the switch port link parameters: speed, duplex, pause based on
401 - ``fixed_link_update``: Function invoked by the PHY library, and specifically by
402   the fixed PHY driver asking the switch driver for link parameters that could
403   not be auto-negotiated, or obtained by reading the PHY registers through MDIO.
405   MoCA or other kinds of non-MDIO managed PHYs where out of band link
409 ------------------
411 - ``get_strings``: ethtool function used to query the driver's strings, will
414 - ``get_ethtool_stats``: ethtool function used to query per-port statistics and
416   RX/TX counters from the network device, with switch driver specific statistics
419 - ``get_sset_count``: ethtool function used to query the number of statistics items
421 - ``get_wol``: ethtool function used to obtain Wake-on-LAN settings per-port, this
423   Wake-on-LAN settings if this interface needs to participate in Wake-on-LAN
425 - ``set_wol``: ethtool function used to configure Wake-on-LAN settings per-port,
428 - ``set_eee``: ethtool function which is used to configure a switch port EEE (Green
429   Ethernet) settings, can optionally invoke the PHY library to enable EEE at the
430   PHY level if relevant. This function should enable EEE at the switch port MAC
431   controller and data-processing logic
433 - ``get_eee``: ethtool function which is used to query a switch port EEE settings,
434   this function should return the EEE state of the switch port MAC controller
435   and data-processing logic as well as query the PHY for its currently configured
438 - ``get_eeprom_len``: ethtool function returning for a given switch the EEPROM
441 - ``get_eeprom``: ethtool function returning for a given switch the EEPROM contents
443 - ``set_eeprom``: ethtool function writing specified data to a given switch EEPROM
445 - ``get_regs_len``: ethtool function returning the register length for a given
446   switch
448 - ``get_regs``: ethtool function returning the Ethernet switch internal register
449   contents. This function might require user-land code in ethtool to
450   pretty-print register values and registers
453 ----------------
455 - ``suspend``: function invoked by the DSA platform device when the system goes to
456   suspend, should quiesce all Ethernet switch activities, but keep ports
457   participating in Wake-on-LAN active as well as additional wake-up logic if
460 - ``resume``: function invoked by the DSA platform device when the system resumes,
461   should resume all Ethernet switch activities and re-configure the switch to be
464 - ``port_enable``: function invoked by the DSA slave network device ndo_open
466   fully enabling a given switch port. DSA takes care of marking the port with
470 - ``port_disable``: function invoked by the DSA slave network device ndo_close
472   fully disabling a given switch port. DSA takes care of marking the port with
477 ------------
479 - ``port_bridge_join``: bridge layer function invoked when a given switch port is
480   added to a bridge, this function should be doing the necessary at the switch
484 - ``port_bridge_leave``: bridge layer function invoked when a given switch port is
486   switch level to deny the leaving port from ingress/egress traffic from the
488   out at the switch hardware for the switch to (re) learn MAC addresses behind
491 - ``port_stp_state_set``: bridge layer function invoked when a given switch port STP
492   state is computed by the bridge layer and should be propagated to switch
493   hardware to forward/block/learn traffic. The switch driver is responsible for
498 ---------------------
500 - ``port_vlan_filtering``: bridge layer function invoked when the bridge gets
505   VLAN ID map/rules.  If there is no PVID programmed into the switch port,
506   untagged frames must be rejected as well. When turned off the switch must
510 - ``port_vlan_prepare``: bridge layer function invoked when the bridge prepares the
512   by the hardware, this function should return ``-EOPNOTSUPP`` to inform the bridge
516 - ``port_vlan_add``: bridge layer function invoked when a VLAN is configured
517   (tagged or untagged) for the given switch port
519 - ``port_vlan_del``: bridge layer function invoked when a VLAN is removed from the
520   given switch port
522 - ``port_vlan_dump``: bridge layer function invoked with a switchdev callback
526 - ``port_fdb_add``: bridge layer function invoked when the bridge wants to install a
527   Forwarding Database entry, the switch hardware should be programmed with the
530   function should return ``-EOPNOTSUPP`` to inform the bridge code to fallback to
534         of DSA, would be its port-based VLAN, used by the associated bridge device.
536 - ``port_fdb_del``: bridge layer function invoked when the bridge wants to remove a
537   Forwarding Database entry, the switch hardware should be programmed to delete
541 - ``port_fdb_dump``: bridge layer function invoked with a switchdev callback
545 - ``port_mdb_prepare``: bridge layer function invoked when the bridge prepares the
547   this function should return ``-EOPNOTSUPP`` to inform the bridge code to fallback
551 - ``port_mdb_add``: bridge layer function invoked when the bridge wants to install
552   a multicast database entry, the switch hardware should be programmed with the
557         of DSA, would be its port-based VLAN, used by the associated bridge device.
559 - ``port_mdb_del``: bridge layer function invoked when the bridge wants to remove a
560   multicast database entry, the switch hardware should be programmed to delete
564 - ``port_mdb_dump``: bridge layer function invoked with a switchdev callback
572 -------------------------------------------------------------
575 capable hardware, but does not enforce a strict switch device driver model. On
577 of the switch specific. At some point we should envision a merger between these
581 --------------------
583 - making the number of ports fully dynamic and not dependent on ``DSA_MAX_PORTS``
584 - allowing more than one CPU/management interface:
586 - porting more drivers from other vendors: