Translating

Before starting, it’s important to look at the translator API and ConfigTree. It may look like a lot and some parts may not make sense at this point but it will help understanding the tutorial.

We are going to see how to translate configuration by example. To do that we are going to use a modified/simplified version of the openconfig-interfaces and openconfig-vlan models. Note they have been slightly modified for simplicity and brevity.

Let’s start by looking at the ASCII tree representation of the model:

+--rw openconfig-interfaces:interfaces
|  +--rw interface* [name]
|     +--rw config
|     |  +--rw description? <string>
|     |  +--rw enabled? <boolean>
|     |  +--rw loopback-mode? <boolean>
|     |  +--rw mtu? <uint16>
|     |  +--rw name? <string>
|     |  +--rw type <identityref>
|     +--rw name <leafref>
|     +--rw subinterfaces
|        +--rw subinterface* [index]
|           +--rw config
|           |  +--rw description? <string>
|           |  +--rw enabled? <boolean>
|           |  +--rw index? <uint32>
|           +--rw index <leafref>
+--rw openconfig-vlan:vlans
   +--rw vlan* [vlan-id]
      +--rw config
      |  +--rw name? <string>
      |  +--rw status? <enumeration>
      |  +--rw vlan-id? <vlan-id(uint16)>
      +--rw vlan-id <leafref>

To write a translator you are going to need to write a class following the rules below:

1. A grouping (either a list or a container) is represented by a class that inherits from yangify.translator.Translator.
2. Classes that implement a part of the tree are nested in the parent object and are named as the grouping it implements.
3. Each class may have a nested class named Yangify that inherits from yangify.translator.TranslatorData. This class may implement code to help with the processing. See the API documentation for details.
4. Finally, leaves are processed by a function named { leaf_name } and will have to to modify the self.yy.result or self.yy.root_result accordingly.

Translator code is going to look very similar to parser code, so refer to the parsing tutorial for details about it.

The openconfig-interfaces translator

To explain how this works we are going to write a translator that will turn a JSON blob following the openconfig models and translate it into IOS configuration. Let’s look at the object first:

%cat data/ios/data.json

{
  "openconfig-interfaces:interfaces": {
    "interface": [
      {
        "name": "FastEthernet1",
        "config": {
          "name": "FastEthernet1",
          "type": "iana-if-type:ethernetCsmacd",
          "description": "This is Fa1",
          "enabled": false
        },
        "subinterfaces": {
          "subinterface": [
            {
              "index": 1,
              "config": {
                "index": 1,
                "description": "This is Fa1.1"
              }
            },
            {
              "index": 2,
              "config": {
                "index": 2,
                "description": "This is Fa1.2"
              }
            }
          ]
        }
      },
      {
        "name": "FastEthernet3",
        "config": {
          "name": "FastEthernet3",
          "type": "iana-if-type:ethernetCsmacd",
          "description": "This is Fa3",
          "enabled": true
        }
      },
      {
        "name": "FastEthernet4",
        "config": {
          "name": "FastEthernet4",
          "type": "iana-if-type:ethernetCsmacd",
          "enabled": false
        }
      }
    ]
  },
  "openconfig-vlan:vlans": {
    "vlan": [
      {
        "vlan-id": 10,
        "config": {
          "vlan-id": 10,
          "name": "prod",
          "status": "ACTIVE"
        }
      },
      {
        "vlan-id": 20,
        "config": {
          "vlan-id": 20,
          "name": "dev",
          "status": "SUSPENDED"
        }
      }
    ]
  }
}

Nothing very complex, just a few interfaces and subinterfaces and a couple of vlans. The code for this tutorial is in tutorial_translator.py, so let’s start by importing it:

import tutorial_translator

Now that we have imported the code let’s start looking at the code piece by piece. The starting point is going to be the class Interfaces which will be used to parse openconfig-interfaces:interfaces:

show_code(tutorial_translator.Interfaces)

class Interfaces(Translator):
    """
    Implements openconfig-interfaces:interfaces

    Using a :obj:`yangify.translator.config_tree.ConfigTree` object for the result
    """

    interface = Interface

Not much here other than the translator explaining how to store the commands. Other than that, the container openconfig-interfaces:interfaces only has a YANG list in the interface node. As interface is a grouping a different Translator class is used. Let’s look at it:

show_code(tutorial_translator.Interface)

class Interface(Translator):
    """
    Implements openconfig-interfaces:interfaces/interface
    """

    class Yangify(TranslatorData):
        def _remove_subinterfaces(self, interface: Dict[str, Any]) -> None:
            """
            A helper function to remove subinterfaces.
            """
            subifaces = interface.get("subinterfaces", {}).get("subinterface", [])
            for subiface in subifaces:
                self.root_result.add_command(
                    f"no interface {self.key}.{subiface['index']}"
                )

        def pre_process_list(self) -> None:
            """
            If we have interfaces to remove we do so before processing the list of interfaces.
            """
            for element in self.to_remove:
                self.result.add_command(f"default interface {element['name']}")
                self._remove_subinterfaces(element)

        def pre_process(self) -> None:
            """
            Before processing a given interface we are going to do a couple of things:

            1. if we are replacing the configuration we default the interface and its
               subinterfaces
            2. We create a placeholder for the interface configuration and we set it
               in self.result
            """
            if self.replace:
                self.root_result.add_command(f"default interface {self.key}")
                if self.running is not None:
                    # self.running.goto(self.path).value is going to return the running
                    # value of the current interface
                    try:
                        self._remove_subinterfaces(self.running.goto(self.path).value)
                    except NonexistentInstance:
                        # if it's a candidate interface self.running.goto(self.path) will
                        # raise this exception
                        pass
            path = f"interface {self.key}"
            # we insert it as soon as possible to maintain order
            self.result = self.root_result.new_section(path)

        def post_process(self) -> None:
            """
            After we are doing processing the interface we can either
            remove entirely the interface from the configuration if it's empty
            or add_command exit\n! to the commands
            """
            path = f"interface {self.key}"
            if not self.result:
                self.root_result.pop_section(path)
            else:
                self.result.add_command("   exit\n!")

    name = unneeded

    subinterfaces = Subinterfaces
    config = InterfaceConfig

Ok, this may look daunting but it’s not that hard. Half of the code is just documentation and comments trying to explain what’s going on. I suggest you to read it carefully but in broad terms:

1. Before parsing the list we remove the interfaces that are no longer needed (assuming a merge or replace operation)
2. Before parsing an element of the list:
3. we default the interface and its subinterfaces if we are doing a replace operation
4. we create a placeholder for our interface configuration in self.yy.result, we also attach the placeholder to the self.yy.root_result
5. Finally, when we are done parsing the interface we either append exit\n! to make it look more like the original IOS configuration or we completely remove the interface from the configuration if it’s empty. This last step will be useful to have cleaner merge operations as we will see later.

Now let’s look at the InterfaceConfig class that implements openconfig-interfaces:interfaces/interface/config:

show_code(tutorial_translator.InterfaceConfig)

class InterfaceConfig(Translator):
    """
    Implements openconfig-interfaces:interfaces/interface/config
    """

    name = unneeded
    type = unneeded

    def description(self, value: Optional[str]) -> None:
        if value:
            self.yy.result.add_command(f"   description {value}")
        else:
            self.yy.result.add_command(f"   no description")

    def enabled(self, value: Optional[bool]) -> None:
        if value:
            self.yy.result.add_command(f"   no shutdown")
        else:
            self.yy.result.add_command(f"   shutdown")

Let’s explain what’s going on here:

1. There is nothing to do with name and type as that doesn’t translate into anything.
2. Both description and enabled are going to receive the new value, which may be empty when merging configuration as we try to unset any of those. All that those methods need to do is append the right command for both setting and unsetting such value.

Now let’s circle back to openconfig-interfaces:interfaces/interface and head down to the subinterfaces container, which was parsed with the Subinterfaces class:

show_code(tutorial_translator.Subinterfaces)

class Subinterfaces(Translator):
    """
    Implements openconfig-interfaces:interfaces/interface/subinterfaces
    """

    subinterface = Subinterface

Like the Interface class, not much to look at here, let’s head down to the Subinterface class:

show_code(tutorial_translator.Subinterface)

class Subinterface(Translator):
    """
    Implements openconfig-interfaces:interfaces/interface/subinterfaces/subinterface
    """

    class Yangify(TranslatorData):
        def pre_process_list(self) -> None:
            """
            If we need to remove itnerfaces we do it here. However, will need to
            get the key of the parent interface first as we will need it
            to remove the subinterfaces. Remember that subinterfaces in openconfig
            are referenced by their index and don't have a fully qualified name
            """
            parent_key = self.keys["/openconfig-interfaces:interfaces/interface"]
            for element in self.to_remove:
                iface_name = f"{parent_key}.{element['index']}"
                self.root_result.add_command(f"no interface {iface_name}")

        def pre_process(self) -> None:
            """
            We create a placeholder for our configuration in self.result, we attach it to
            self.root_result and also default the subinterface if we are in replace mode
            """
            parent_key = self.keys["/openconfig-interfaces:interfaces/interface"]
            self.keys["subinterface_key"] = f"{parent_key}.{self.key}"
            path = f"interface {self.keys['subinterface_key']}"
            if self.replace:
                self.root_result.add_command(
                    f"no interface {self.keys['subinterface_key']}"
                )
            self.result = self.root_result.new_section(path)

        def post_process(self) -> None:
            """
            After we are doing processing the interface we can either
            remove entirely the interface from the configuration if it's empty
            or add_command exit\n! to the commands
            """
            path = f"interface {self.keys['subinterface_key']}"
            if not self.result:
                self.root_result.pop_section(path)
            else:
                self.result.add_command("   exit\n!")

    index = unneeded
    config = SubinterfaceConfig

Like the Interface class it might look daunting but it shouldn’t be that hard to grasp. It’s also very similar to it. I suggest reading the code carefully, paying attention to the commands and referring back to the explanation after the Interface class if something is not clear.

Finally, let’s look at the SubinterfaceConfig class:

show_code(tutorial_translator.SubinterfaceConfig)

class SubinterfaceConfig(Translator):
    """
    Implements openconfig-interfaces:interfaces/interface/subinterfaces/subinterface/config
    """

    index = unneeded

    def description(self, value: Optional[str]) -> None:
        if value:
            self.yy.result.add_command(f"   description {value}")
        else:
            self.yy.result.add_command(f"   no description")

Almost ideantical to the InterfaceConfig class.

Now that we have the Translator classes we need to create the root translator. The root translator has the following functions:

1. Allow the user pick and choose which translators to use
2. Initialize the results, if needed.
3. Perform some post operations, if needed.

Using the translator

Our root class is going to load the Interfaces translator:

from yangify import translator
from yangify.translator.config_tree import ConfigTree


class IOSTranslator(translator.RootTranslator):
    class Yangify(translator.TranslatorData):
        def init(self) -> None:
            self.root_result = ConfigTree()
            self.result = self.root_result

        def post(self) -> None:
            self.root_result = self.root_result.to_string()

    interfaces = tutorial_translator.Interfaces

Now we need to load the data:

import json
with open("data/ios/data.json", "r") as f:
    data = json.load(f)

Now we will create the datamodel as we will need it later on:

from yangson.datamodel import DataModel
dm = DataModel.from_file("yang/yang-library-data.json", ["yang/yang-modules/ietf", "yang/yang-modules/openconfig"])

Finally, we are going to instantiate the IOSTranslator and call the process method:

p = IOSTranslator(dm, candidate=data)
result = p.process()

Now that we got the processed object, let’s see the result:

print(result)

interface FastEthernet1
   description This is Fa1
   shutdown
   exit
!
interface FastEthernet1.1
   description This is Fa1.1
   exit
!
interface FastEthernet1.2
   description This is Fa1.2
   exit
!
interface FastEthernet3
   description This is Fa3
   no shutdown
   exit
!
interface FastEthernet4
   shutdown
   exit
!

Adding a second translator

In the previous example we created a parser that only translates the openconfig-interfaces model, however, our tutorial_translator.py contains code to also translate the openconfig-vlan model, let’s create a second RootTranslate class that can translate both models:

from yangify import translator
from yangify.translator.config_tree import ConfigTree


class IOSTranslator2(translator.RootTranslator):
    class Yangify(translator.TranslatorData):
        def init(self) -> None:
            self.root_result = ConfigTree()
            self.result = self.root_result

        def post(self) -> None:
            self.root_result = self.root_result.to_string()

    interfaces = tutorial_translator.Interfaces
    vlans = tutorial_translator.Vlans

p = IOSTranslator2(dm, candidate=data)
result = p.process()

print(result)

interface FastEthernet1
   description This is Fa1
   shutdown
   exit
!
interface FastEthernet1.1
   description This is Fa1.1
   exit
!
interface FastEthernet1.2
   description This is Fa1.2
   exit
!
interface FastEthernet3
   description This is Fa3
   no shutdown
   exit
!
interface FastEthernet4
   shutdown
   exit
!
vlan 10
   name prod
   no shutdown
   exit
!
vlan 20
   name dev
   shutdown
   exit
!

Replace

Yangify implements a replace mode. The replace mode is useful to perform “partial replaces”, let’s see that last example enabling the replace functionality:

p = IOSTranslator2(dm, candidate=data, replace=True)
result = p.process()

print(result)

default interface FastEthernet1
interface FastEthernet1
   description This is Fa1
   shutdown
   exit
!
no interface FastEthernet1.1
interface FastEthernet1.1
   description This is Fa1.1
   exit
!
no interface FastEthernet1.2
interface FastEthernet1.2
   description This is Fa1.2
   exit
!
default interface FastEthernet3
interface FastEthernet3
   description This is Fa3
   no shutdown
   exit
!
default interface FastEthernet4
interface FastEthernet4
   shutdown
   exit
!
no vlan 10
vlan 10
   name prod
   no shutdown
   exit
!
no vlan 20
vlan 20
   name dev
   shutdown
   exit
!

As you can see, what’s happening here is that each block is being defaulted to it’s original config (by either using the default or no prefix) before applying the configuration for such block. This is useful to clean those blocks and remove configuration that was applied manually and is not covered by your YANG models.