Parsing Quick Start Guide¶
This quick start walks through creating YANGify parsers for both LLDP neighbors and interfaces. The goal of this is to take native data, such as show commands, from a traditional network device and convert it to structured data (JSON) that adheres to a given YANG model.
Parsing LLDP Neighbors & Interfaces¶
Our example will use a Cisco IOS device and the goal is to represent and create LLDP and interface data that adheres to OpenConfig YANG models. The OC model for LLDP actually accounts for quite a bit of data from basic LLDP configuration of a device, to its neighbors, to low level details such as counters, timers, and TTL information often found in a combination of show commands including show run, show lldp neighbors, and show lldp neighbor detail just to name a few. The point is
multiple commands are required to generate and fully populate the data required for a complete YANG model. It’s often unnecessary to worry about every feature knob, so in this exercise, we are only going to worry about key aspects of LLDP being very much focused on LLDP neighbors of a given device. The reason we’ll also be parsing interfaces is that the LLDP model actually makes references to interfaces that must be valid and exist on the device, so to properly parse LLDP, we need to parse
interfaces too.
Getting Familiar with Data¶
Before we get started, it’s quite helpful to visually see the data that we’ll be working with.
Let’s look at the starting config along with the tree output of the YANG model we’re generating data for, and the JSON representations of the data that we’ll be creating with Yangify parsers.
Native Configuration and Operational Data¶
View the baseline configuration:
[1]:
%cat ../data/ios/ntc-r1/config.txt
hostname ntc-r1
!
interface GigabitEthernet1
description Hello GigE1
shutdown
!
interface GigabitEthernet2
description Hello GigE2
!
interface GigabitEthernet3
description Hello GigE3
!
interface GigabitEthernet4
description Hello GigE4
no lldp enable
!
interface GigabitEthernet5
description Hello GigE5
!
interface Loopback100
description this is loopy100
!
vlan 10
vlan 20
name web_vlan
vlan 30
name test_vlan
!
View the LLDP neighbors output:
[2]:
%cat ../data/ios/ntc-r1/lldp.txt
Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other
Device ID Local Intf Hold-time Capability Port ID
ntc-r2 Gi2 120 B Gi0/1
ntc-s2 Gi2 120 B Gi0/1
ntc-r3 Gi3 120 B,R Gi0/3
ntc-r5 Gi3 120 B,R Gi0/4
ntc-r6 Gi1 120 B,R Gi0/4
ntc-r7 Gi1 120 B,R Gi0/2
ntc-r8 Gi1 120 B,R Gi0/3
Total entries displayed: 7
Let’s also gain some insight into what the OpenConfig YANG model looks like for LLDP neighbors.
Tree Output of the OpenConfig LLDP YANG Model¶
Note: you can installpyangwithpip.
ntc@nautobot:models (master)$ pyang -f tree lldp/openconfig-lldp.yang
module: openconfig-lldp
+--rw lldp
+--rw config
| +--rw enabled? boolean
| +--rw hello-timer? uint64
| +--rw suppress-tlv-advertisement* identityref
| +--rw system-name? string
| +--rw system-description? string
| +--rw chassis-id? string
| +--rw chassis-id-type? oc-lldp-types:chassis-id-type
+--ro state
| +--ro enabled? boolean
| +--ro hello-timer? uint64
| +--ro suppress-tlv-advertisement* identityref
| +--ro system-name? string
| +--ro system-description? string
| +--ro chassis-id? string
| +--ro chassis-id-type? oc-lldp-types:chassis-id-type
| +--ro counters
| +--ro frame-in? yang:counter64
| +--ro frame-out? yang:counter64
| +--ro frame-error-in? yang:counter64
| +--ro frame-discard? yang:counter64
| +--ro tlv-discard? yang:counter64
| +--ro tlv-unknown? yang:counter64
| +--ro last-clear? yang:date-and-time
| +--ro tlv-accepted? yang:counter64
| +--ro entries-aged-out? yang:counter64
+--rw interfaces
+--rw interface* [name]
+--rw name -> ../config/name
+--rw config
| +--rw name? oc-if:base-interface-ref
| +--rw enabled? boolean
+--ro state
| +--ro name? oc-if:base-interface-ref
| +--ro enabled? boolean
| +--ro counters
| +--ro frame-in? yang:counter64
| +--ro frame-out? yang:counter64
| +--ro frame-error-in? yang:counter64
| +--ro frame-discard? yang:counter64
| +--ro tlv-discard? yang:counter64
| +--ro tlv-unknown? yang:counter64
| +--ro last-clear? yang:date-and-time
| +--ro frame-error-out? yang:counter64
+--ro neighbors
+--ro neighbor* [id]
+--ro id -> ../state/id
+--ro config
+--ro state
| +--ro system-name? string
| +--ro system-description? string
| +--ro chassis-id? string
| +--ro chassis-id-type? oc-lldp-types:chassis-id-type
| +--ro id? string
| +--ro age? uint64
| +--ro last-update? int64
| +--ro ttl? uint16
| +--ro port-id? string
| +--ro port-id-type? oc-lldp-types:port-id-type
| +--ro port-description? string
| +--ro management-address? string
| +--ro management-address-type? string
+--ro custom-tlvs
| +--ro tlv* [type oui oui-subtype]
| +--ro type -> ../state/type
| +--ro oui -> ../state/oui
| +--ro oui-subtype -> ../state/oui-subtype
| +--ro config
| +--ro state
| +--ro type? int32
| +--ro oui? string
| +--ro oui-subtype? string
| +--ro value? binary
+--ro capabilities
+--ro capability* [name]
+--ro name -> ../state/name
+--ro config
+--ro state
+--ro name? identityref
+--ro enabled? boolean
ntc@nautobot:models (master)$
For this exercise, we are only interested in a sub-set of the data from the full model. You get to choose exactly what parts of the model you want to parse with Yangify.
This is the sub-set of the YANG model our script and parsers will generate data for:
ntc@nautobot:models (master)$ pyang -f tree lldp/openconfig-lldp.yang
module: openconfig-lldp
+--rw lldp
+--rw config
| +--rw system-name? string
+--rw interfaces
+--rw interface* [name]
+--rw name -> ../config/name
+--rw config
| +--rw name? oc-if:base-interface-ref
+--ro neighbors
+--ro neighbor* [id]
+--ro id -> ../state/id
+--ro state
| +--ro id? string
| +--ro port-id? string
ntc@nautobot:models (master)$
And now a glimpse into native YANG:
Naive YANG Output¶
container lldp {
description
"Top-level container for LLDP configuration and state data";
container config { // this is the container that'll hold the system-name
description
"Configuration data ";
// shortened for brevity
JSON Representation of OpenConfig LLDP YANG Data¶
Most importantly, here is JSON data of what we’ll be building given the sub-set of the model in question. This is the whole premise behind Yangify. It’s to provide a framework that makes it much easier to parse (and translate) data that maps directly back to YANG models. You should be able to cross-reference the ASCII tree back to the JSON based on the keys used in the following output:
{
"openconfig-lldp:lldp": {
"config": {
"system-name": "ntc-r1"
},
"interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1"
},
"neighbors": {
"neighbor": [
{
"id": "ntc-r6",
"state": {
"id": "ntc-r6",
"port-id": "Gi0/4"
}
}
]
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2"
},
"neighbors": {
"neighbor": [
{
"id": "ntc-r2",
"state": {
"id": "ntc-r2",
"port-id": "Gi0/1"
}
},
{
"id": "ntc-s2",
"state": {
"id": "ntc-s2",
"port-id": "Gi0/1"
}
}
]
}
}
]
}
}
}
That covers the data we’ll be starting with as well as a view into what will be generated.
Parsing Configurations¶
First, we’ll look at how Yangify parses native “traditional CLI” configs.
As a reminder the following is our starting (partial configuration).
[3]:
%cat ../data/ios/ntc-r1/config.txt
hostname ntc-r1
!
interface GigabitEthernet1
description Hello GigE1
shutdown
!
interface GigabitEthernet2
description Hello GigE2
!
interface GigabitEthernet3
description Hello GigE3
!
interface GigabitEthernet4
description Hello GigE4
no lldp enable
!
interface GigabitEthernet5
description Hello GigE5
!
interface Loopback100
description this is loopy100
!
vlan 10
vlan 20
name web_vlan
vlan 30
name test_vlan
!
Let’s get famililar with the text tree parser, the parse_intended_config function, used by Yangify.
[4]:
import json
from yangify.parser.text_tree import parse_indented_config
with open("../data/ios/ntc-r1/config.txt", "r") as f:
config = f.read()
parsed_config = parse_indented_config(config.splitlines())
print(json.dumps(parsed_config, indent=4))
{
"#list": [],
"#text": "",
"hostname": {
"#text": "ntc-r1",
"ntc-r1": {
"#standalone": true
}
},
"interface": {
"#text": "Loopback100",
"GigabitEthernet1": {
"#list": [
{
"description": {
"#text": "Hello GigE1",
"Hello": {
"#text": "GigE1",
"GigE1": {
"#standalone": true
}
}
}
}
],
"#text": "shutdown",
"description": {
"#text": "Hello GigE1",
"Hello": {
"#text": "GigE1",
"GigE1": {
"#standalone": true
}
}
},
"shutdown": {
"#standalone": true
},
"#standalone": true
},
"GigabitEthernet2": {
"#list": [
{
"description": {
"#text": "Hello GigE2",
"Hello": {
"#text": "GigE2",
"GigE2": {
"#standalone": true
}
}
}
}
],
"#text": "description Hello GigE2",
"description": {
"#text": "Hello GigE2",
"Hello": {
"#text": "GigE2",
"GigE2": {
"#standalone": true
}
}
},
"#standalone": true
},
"GigabitEthernet3": {
"#list": [
{
"description": {
"#text": "Hello GigE3",
"Hello": {
"#text": "GigE3",
"GigE3": {
"#standalone": true
}
}
}
}
],
"#text": "description Hello GigE3",
"description": {
"#text": "Hello GigE3",
"Hello": {
"#text": "GigE3",
"GigE3": {
"#standalone": true
}
}
},
"#standalone": true
},
"GigabitEthernet4": {
"#list": [
{
"description": {
"#text": "Hello GigE4",
"Hello": {
"#text": "GigE4",
"GigE4": {
"#standalone": true
}
}
}
},
{
"no": {
"#text": "lldp enable",
"lldp": {
"#text": "enable",
"enable": {
"#standalone": true
}
}
}
}
],
"#text": "no lldp enable",
"description": {
"#text": "Hello GigE4",
"Hello": {
"#text": "GigE4",
"GigE4": {
"#standalone": true
}
}
},
"no": {
"#text": "lldp enable",
"lldp": {
"#text": "enable",
"enable": {
"#standalone": true
}
}
},
"#standalone": true
},
"GigabitEthernet5": {
"#list": [
{
"description": {
"#text": "Hello GigE5",
"Hello": {
"#text": "GigE5",
"GigE5": {
"#standalone": true
}
}
}
}
],
"#text": "description Hello GigE5",
"description": {
"#text": "Hello GigE5",
"Hello": {
"#text": "GigE5",
"GigE5": {
"#standalone": true
}
}
},
"#standalone": true
},
"Loopback100": {
"#list": [
{
"description": {
"#text": "this is loopy100",
"this": {
"#text": "is loopy100",
"is": {
"#text": "loopy100",
"loopy100": {
"#standalone": true
}
}
}
}
}
],
"#text": "description this is loopy100",
"description": {
"#text": "this is loopy100",
"this": {
"#text": "is loopy100",
"is": {
"#text": "loopy100",
"loopy100": {
"#standalone": true
}
}
}
},
"#standalone": true
}
},
"": {
"#standalone": true
},
"vlan": {
"#text": "30",
"10": {
"#standalone": true
},
"20": {
"#list": [
{
"name": {
"#text": "web_vlan",
"web_vlan": {
"#standalone": true
}
}
}
],
"#text": "name web_vlan",
"name": {
"#text": "web_vlan",
"web_vlan": {
"#standalone": true
}
},
"#standalone": true
},
"30": {
"#list": [
{
"name": {
"#text": "test_vlan",
"test_vlan": {
"#standalone": true
}
}
}
],
"#text": "name test_vlan",
"name": {
"#text": "test_vlan",
"test_vlan": {
"#standalone": true
}
},
"#standalone": true
}
}
}
You can find more information about this parser in the API documentation, but it’s essentially a smart parser that creates a dictionary for each command set (or command) while also understanding hierarchy. For commands with multiple words on the same line, it creates a nested dictionary per word with the final value at the #text key; and a key with the same as the value, with #standalone for when it’s the last word too.
Using Yangify¶
The first step is to create a script that we’ll use to consume the parsers we’re building. Our script will just print out the generated structured data. Using it beyond this for network operations is out of scope of this guide.
[5]:
import json
from yangify import parser
from yangify.parser.text_tree import parse_indented_config
from yangson.datamodel import DataModel
import task1_lldp_parser
dm = DataModel.from_file("../yang/yang-library-data.json", ["../yang/yang-modules/ietf", "../yang/yang-modules/openconfig"])
class IOSParser(parser.RootParser):
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = {}
self.root_native["show run"] = parse_indented_config(
self.native["show run"].splitlines()
)
self.native = self.root_native
# interfaces is YANG container. Variable names are always
# YANG containers or lists
interfaces = task1_lldp_parser.Interfaces
with open("../data/ios/ntc-r1/config.txt", "r") as f:
config = f.read()
data = {}
data["show run"] = config
Few things to note in the script so far:
You’ll always need to instantiate a
DataModelobject as shown at the top of the script with the variabledm. You’ll need to pass in a JSON file that is a library of your available models (per an RFC) and then a list of available paths Yangify will search for those models.You’ll always start with a base class that has an arbritrary name inheiriting
parser.RootParserwith a nested class with a special class name ofYangify. Here you’ll always set two propertiesroot_nativeandnativethat’ll be used as the data that’ll be parsed. The object types of both of these variables don’t matter, but it’ll be common to use a dictionary just in case we need to pass in data from multiple show commands (or API calls) into the Parser.The variable above called
interfacesis the outer most element being built by the parser. In YANG speak,interfacesis a YANG container, and this is how you’ll likely start most of the development when using Yangify, rooting the parsing at a specific contaienr.Note:
interfacesis a YANG container.
We’ve mentioned root_native and native, but what exactly are they?
root_native should be and will always be the original starting point of the native configuration. We’ll soon see this could be text or any other data type. In fact, we’ll start with a show run for our example and add a dictionary to it later on. When things are kicked off, we are setting native to be equal to root_native, so they are the same when parsing begins. As parsing takes place, native does transform within the parser as lists are processed. We will see this
shortly.
Building out Yangify Parsers¶
Before we take a look at the task1_lldp_parser Python module, let’s let’s take quick look at the abbreviated YANG ASCII tree for interfaces that we are using for the tutorial:
ntc@nautobot:openconfig (lldp-dev)$ pyang -f tree openconfig-interfaces.yang
module: openconfig-interfaces
+--rw interfaces
+--rw interface* [name] // yang list
+--rw name -> ../config/name
+--rw config
| +--rw name? string
| +--rw type identityref
| +--rw mtu? uint16
| +--rw loopback-mode? boolean
| +--rw description? string
| +--rw enabled? boolean
+--rw subinterfaces
+--rw subinterface* [index]
+--rw index -> ../config/index
+--rw config
+--rw index? uint32
+--rw description? string
+--rw enabled? boolean
Take note of the structure: it’s interfaces (YANG Container) -> interface (YANG list with key of name) -> a YANG leaf of name, and then another container called config. We won’t be using anything else shown for the parsing.
Keep in mind, we already referenced the interfaces container in our starting script with the following statement:
interfaces = task1_lldp_parser.Interfaces
This tells us the first class we’ll need to create is called Interfaces in the task1_lldp_parser module.
Let’s dive into parser.
While the actual goal is to build a parser for LLDP neighbors, the LLDP neighbors OpenConfig YANG model references the OpenConfig interfaces model ensuring valid interfaces are being used in the LLDP model. This can be seen by a specific line in the LLDP model–refer back to the ASCII tree:
+--rw name? oc-if:base-interface-ref
Once we finish parsing interfaces, we’ll move onto LLDP.
Let’s now instantiate the IOSParser and test the initial parser (yet to be covered).
We are passing in two arguments below to get us going. The positional argument of the DataModel built earlier and then data which contains what we’ll parse.
Note: when you execute the next step, you’re supposed to get an error.
[6]:
from yangson.exceptions import SchemaError
p = IOSParser(dm, native=data)
try:
result = p.process()
print(json.dumps(result.raw_value(), indent=4))
except SchemaError as e:
print("Supressing output for readability:")
print(f"error: {e}")
Supressing output for readability:
error: [/openconfig-interfaces:interfaces/interface/0] missing-data: expected 'config'
This script uses a parser called task1_lldp_parser as you may have noticed in the imports. You can tell it didn’t actually work. Let’s start to explore the parser and see why it failed.
[7]:
%cat task1_lldp_parser.py
from typing import Any, Dict, Iterator, Tuple
from yangify.parser import Parser, ParserData
class Interface(Parser):
"""
Implements openconfig-interfaces:interfaces/interface
"""
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text" or "." in interface:
continue
yield interface, data
def name(self) -> str:
return self.yy.key
class Interfaces(Parser):
"""
Implements openconfig-interfaces:interfaces
"""
interface = Interface
Parser Classes¶
Reviewing the parser, the first class to build is the Interfaces class since we already had it in our script.
Few things to note as you look at the parser above:
- Class names ARE arbitrary.
- Variable names ARE NOT arbitrary.
- Variable names are either YANG containers or lists.
- Methods in each Class are YANG leaf objects
With that said, the class has a variable called interface. This is the YANG list that contains the interfaces. We’ve assigned that variable a value of Interface and if we look in that class we see a method called name which is a YANG leaf. You can start to conceptualize the model with the variable and method names by cross-referencing the YANG tree output we saw earlier.
Don’t worry, we’ll go through the parser in more detail in just a minute.
Okay, so why didn’t the processing work the first time? It’s because we violated the YANG model.
Recall this part of the tree output:
module: openconfig-interfaces
+--rw interfaces
+--rw interface* [name] // yang list
+--rw name -> ../config/name
+--rw config
| +--rw name? string
| +--rw type identityref
Next to a leaf, there is a ? if it’s optional. We can see that there is a leaf called type that is required (no question mark), thus the config container is required.
Let’s process the parser again, but this time disable YANG validation.
[8]:
p = IOSParser(dm, native=data)
result = p.process(validate=False)
print(json.dumps(result.raw_value(), indent=4))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1"
},
{
"name": "GigabitEthernet2"
},
{
"name": "GigabitEthernet3"
},
{
"name": "GigabitEthernet4"
},
{
"name": "GigabitEthernet5"
},
{
"name": "Loopback100"
}
]
}
}
This time it worked because we disabled validation meaning it didn’t work the first time because the config container is a required container for each interface.
Let’s update the parser, now with a new name, to do more testing while adding the config container with a few leaves.
[9]:
%cat task2_lldp_parser.py
from typing import Any, Dict, Iterator, Tuple
from yangify.parser import Parser, ParserData
class InterfaceConfig(Parser):
"""
Implements openconfig-interfaces:interfaces/interface/config
"""
def name(self) -> str:
return self.yy.key
def enabled(self) -> bool:
shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
if shutdown:
return False
else:
return True
class Interface(Parser):
"""
Implements openconfig-interfaces:interfaces/interface
"""
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text" or "." in interface:
continue
yield interface, data
config = InterfaceConfig
def name(self) -> str:
return self.yy.key
class Interfaces(Parser):
"""
Implements openconfig-interfaces:interfaces
"""
interface = Interface
Although we added a bit to the parser, we still need to understand the built in attributes like yy being used along with the Yangify class and the extract_elements method. Let’s take a look.
Diving Deeper into the Yangify Parsing Logic¶
This is where things get interesting. YANG lists become fun to work with.
Keep in mind, after doing a few of these, the process will become pretty methodical.
So what goes on inside this Interface class?
Let’s look at the variable config and method called name.
By looking at the model and remembering what we described already, we should know config is either a YANG container or list, and based on the model itself, we do in fact know it’s a YANG container. name on the other hand is a YANG leaf because all methods are YANG leaves!
The Yangify Class¶
Since the class we’re inside of, Interface, is representative of a YANG list and the list requires a key denoted by [name] in the ASCII tree output of the model, and that key is a YANG leaf itself, it’s represented as a method inside the class. Again, all method names are YANG leaves.
In order to bring this all together, we need to look at the Yangify class being used inside Interface
You’ll basically use the Yangify class whenever you need to work with the original data we passed into IOSParser. This will be a requirement whenever working with YANG lists because usually data needs be looped over and the method called extract_elements within the Yangify class is a hard-set requirement within the Yangify framework. It must be used for lists.
Parsing YANG Lists¶
In order to extract individual list elements required to build the desired data set, we need to understand the existing data that we can consume. First and foremost, native and root_native are available, e.g. the data we passed in to kick things off in the first place. This data is now consumable as a dictionary created by the parse_intended_config that we talked about earlier.
Accessing Available Data¶
Let’s look at the available data both inside extract_elements and the leaf, e.g. the method called name.
The [YANG] key for the YANG list is name, which is the interface name itself, e.g. GigabitEthernet1 and so on. You can look back and see that the interface names are actually keys in the dictionary created self.native["show run"]["interface"], which is the result of the text tree parser, e.g. result of parse_intended_config. That’s how we’ll access the data.
As the interfaces are looped over and processed, we need to skip any key that is equal to #textdue to an artifact of the parse_intended_config function (as documented in the Yangify docs) and skip any key with a . since that denotes a sub-interface. Sub-interfaces are handled differently per the interfaces YANG model and we are skipping them for now.
As the data (interfaces for now) is looped over, we need to return, or more precisely yield the right data within the extract_elements method so the other methods can consume this data. Using yield allows the function to maintain its state between function calls (unlike returning data with the return statement).
The extract_elements function should always return a tuple that is a key-value pair, the key being the YANG list key and the value being the relevant configuration data for that element.
Within the Interface class as shown above, we know the method called name is the key for the YANG list. It’s shown here:
def name(self) -> str:
return self.yy.key
The neat thing is that inside the Interface class, the key that was yielded from extract_elements, e.g. interface is accessible inside the methods in that class using self.yy.key. Note: You can actually access the value too using self.yy.native. Don’t worry, we’ll use that in an example soon.
This means that self.yy.key in the name method is equivalent to interface that was in the yield statement for each iteration of the loop.
Also take note of the config variable that was added, e.g. YANG container assigned the value of InterfaceConfig. This container is required to be valid data. This new class has 2 methods, each method being a YANG leaf.
Let’s re-instiate IOSParser and generate a new parsed output that has the config container.
[10]:
import task2_lldp_parser
class IOSParser(parser.RootParser):
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = {}
self.root_native["show run"] = parse_indented_config(
self.native["show run"].splitlines()
)
self.native = self.root_native
interfaces = task2_lldp_parser.Interfaces
p = IOSParser(dm, native=data)
result = p.process(validate=False)
print(json.dumps(result.raw_value(), indent=4))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1",
"enabled": false
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2",
"enabled": true
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3",
"enabled": true
}
},
{
"name": "GigabitEthernet4",
"config": {
"name": "GigabitEthernet4",
"enabled": true
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5",
"enabled": true
}
},
{
"name": "Loopback100",
"config": {
"name": "Loopback100",
"enabled": true
}
}
]
}
}
You should also know that this would still fail if we tried to validate the model because from what we said earlier the leaf called type is required.
Let’s add more YANG leafs to the config container.
[11]:
%cat task3_lldp_parser.py
from typing import Any, Dict, Iterator, Optional, Tuple, cast
from yangify.parser import Parser, ParserData
class InterfaceConfig(Parser):
"""
Implements openconfig-interfaces:interfaces/interface/config
"""
def description(self) -> Optional[str]:
return cast(Optional[str], self.yy.native.get("description", {}).get("#text"))
def name(self) -> str:
return self.yy.key
def enabled(self) -> bool:
shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
if shutdown:
return False
else:
return True
def type(self) -> str:
if "Ethernet" in self.yy.key:
return "iana-if-type:ethernetCsmacd"
elif "Loopback" in self.yy.key:
return "iana-if-type:softwareLoopback"
else:
raise ValueError(f"don't know type for interface {self.yy.key}")
def loopback_mode(self) -> bool:
if "Loopback" in self.yy.key:
return True
return False
class Interface(Parser):
"""
Implements openconfig-interfaces:interfaces/interface
"""
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text" or "." in interface:
continue
yield interface, data
config = InterfaceConfig
def name(self) -> str:
return self.yy.key
class Interfaces(Parser):
"""
Implements openconfig-interfaces:interfaces
"""
interface = Interface
This class InterfaceConfig represents each instance of an interface within the interfaces list. Each method in this class is a YANG leaf.
When using lists like interfaces and you have a class that represents an instance like this, note that you still have access to self.yy.
For example, you’ll note the name method in this class is the same as the name method in the Interface class, e.g. they are returning the same value. It’s all about being aware of what data you have access to.
You’ll also note that self.yy.native is used. We alluded to this already. self.yy.native is actually the value in the key-value pair that extract_elements returned within the Interface class. However, it’s not always equal to that value.
Everywhere you see self.yy.native, it is either equal to self.native or equal to the value returned by extract_elements during each loop iteration. It all depends on scoping and what type of data you’re working with.
Let’s re-instantiate IOSParser and generate our parsed output, this time also re-enabling validation.
[12]:
import task3_lldp_parser
class IOSParser(parser.RootParser):
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = {}
self.root_native["show run"] = parse_indented_config(
self.native["show run"].splitlines()
)
self.native = self.root_native
interfaces = task3_lldp_parser.Interfaces
p = IOSParser(dm, native=data)
result = p.process(validate=True)
print(json.dumps(result.raw_value(), indent=4))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE1",
"enabled": false
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE2",
"enabled": true
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE3",
"enabled": true
}
},
{
"name": "GigabitEthernet4",
"config": {
"name": "GigabitEthernet4",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE4",
"enabled": true
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE5",
"enabled": true
}
},
{
"name": "Loopback100",
"config": {
"name": "Loopback100",
"type": "iana-if-type:softwareLoopback",
"loopback-mode": true,
"description": "this is loopy100",
"enabled": true
}
}
]
}
}
Troubleshooting Tip¶
If something isn’t making sense like some of these builtin attributes, just add import pdb; pdb.trace() to the line of interest, so you can use the Python debugger to explore the variables.
Let’s try it:
def enabled(self) -> bool:
import pdb; pdb.trace()
shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
if shutdown:
return False
else:
return True
Run the script.
You’ll see this output:
(yangify) ntc@nautobot:lldp (lldp-dev)$ python lldp-sample.py
IOSParser: doesn't implement ietf-yang-library:modules-state
/openconfig-interfaces:interfaces/interface/config: doesn't implement openconfig-interfaces:mtu
> /ntc-data/repos/yangify/docs/lldp/lldp_parser.py(24)enabled()
-> shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
(Pdb)
Now you can explore:
(Pdb)
(Pdb) print(self.yy.key)
GigabitEthernet1
(Pdb)
(Pdb) import json
(Pdb) print(json.dumps(self.yy.native, indent=4))
{
"#list": [
{
"description": {
"#text": "Hello GigE1",
"Hello": {
"#text": "GigE1",
"GigE1": {
"#standalone": true
}
}
}
}
],
"#text": "shutdown",
"description": {
"#text": "Hello GigE1",
"Hello": {
"#text": "GigE1",
"GigE1": {
"#standalone": true
}
}
},
"shutdown": {
"#standalone": true
},
"#standalone": true
}
(Pdb)
(Pdb) exit
Once you understand the data, it becomes much easier to return the right data, of course. Using pdb is bit cleaner than adding print statements all over the script.
Make sure to remove the ``pdb`` statement.
Parsing LLDP Neighbors¶
At this point, we’re half way there. We’ve parsed interfaces and are ready to begin the parsing LLDP neighbors.
Our interest is to generate data for LLDP very much focused on the YANG container called lldp that we looked at earlier in this tutorial.
This means we need to:
- add the data required for the LLDP parsers to consume
- add the line in the script to create the
lldpcontainer - update
root_nativeto include the new data - update the
taskN_lldp_parserPython module with the appropriate parser classes
Reminder, these are the neigbhors we are working with:
[13]:
%cat ../data/ios/ntc-r1/lldp.txt
Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other
Device ID Local Intf Hold-time Capability Port ID
ntc-r2 Gi2 120 B Gi0/1
ntc-s2 Gi2 120 B Gi0/1
ntc-r3 Gi3 120 B,R Gi0/3
ntc-r5 Gi3 120 B,R Gi0/4
ntc-r6 Gi1 120 B,R Gi0/4
ntc-r7 Gi1 120 B,R Gi0/2
ntc-r8 Gi1 120 B,R Gi0/3
Total entries displayed: 7
Parsing the LLDP data¶
We’ve added the data, but the only native text parser in Yangify is the text tree parser meant for those traditional CLI configurations, not operational data commands like show lldp neighbors. However, there are a number of pre-built TextFSM templates in ntc-templates, we can leverage. Let’s try it, but first make sure you pip install textfsm and do a git clone on ntc-templates.
We can use the following code block to use the TextFSM template and then subsequently transform the data into a more usable structure:
[14]:
import textfsm
with open("../data/ios/ntc-r1/lldp.txt", "r") as f:
lldp_txt = f.read()
template = '../data/ntc-templates/cisco_ios_show_lldp_neighbors.template'
table = textfsm.TextFSM(open(template))
converted_data = table.ParseText(lldp_txt)
[15]:
print(converted_data)
[['ntc-r2', 'Gi2', 'Gi0/1'], ['ntc-s2', 'Gi2', 'Gi0/1'], ['ntc-r3', 'Gi3', 'Gi0/3'], ['ntc-r5', 'Gi3', 'Gi0/4'], ['ntc-r6', 'Gi1', 'Gi0/4'], ['ntc-r7', 'Gi1', 'Gi0/2'], ['ntc-r8', 'Gi1', 'Gi0/3']]
[16]:
neighbors = {}
for item in converted_data:
neighbor = item[0]
local_interface = item[1].replace("Gi", "GigabitEthernet")
neighbor_interface = item[2]
if not neighbors.get(local_interface):
neighbors[local_interface] = []
neighbor = dict(
local_interface=local_interface,
neighbor=neighbor,
neighbor_interface=neighbor_interface,
)
neighbors[local_interface].append(neighbor)
print(json.dumps(neighbors, indent=4))
{
"GigabitEthernet2": [
{
"local_interface": "GigabitEthernet2",
"neighbor": "ntc-r2",
"neighbor_interface": "Gi0/1"
},
{
"local_interface": "GigabitEthernet2",
"neighbor": "ntc-s2",
"neighbor_interface": "Gi0/1"
}
],
"GigabitEthernet3": [
{
"local_interface": "GigabitEthernet3",
"neighbor": "ntc-r3",
"neighbor_interface": "Gi0/3"
},
{
"local_interface": "GigabitEthernet3",
"neighbor": "ntc-r5",
"neighbor_interface": "Gi0/4"
}
],
"GigabitEthernet1": [
{
"local_interface": "GigabitEthernet1",
"neighbor": "ntc-r6",
"neighbor_interface": "Gi0/4"
},
{
"local_interface": "GigabitEthernet1",
"neighbor": "ntc-r7",
"neighbor_interface": "Gi0/2"
},
{
"local_interface": "GigabitEthernet1",
"neighbor": "ntc-r8",
"neighbor_interface": "Gi0/3"
}
]
}
Add the parsed neighbors to data which is the data we actually parse to generate the YANG based JSON.
[17]:
data["lldp_data"] = neighbors
Now let’s add a basic parser for LLDP - note the two new classes at the bottom of the new parser:
[18]:
%cat task4_lldp_parser.py
from typing import Any, Dict, Iterator, Optional, Tuple, cast
from yangify.parser import Parser, ParserData
class InterfaceConfig(Parser):
"""
Implements openconfig-interfaces:interfaces/interface/config
"""
def description(self) -> Optional[str]:
return cast(Optional[str], self.yy.native.get("description", {}).get("#text"))
def name(self) -> str:
return self.yy.key
def enabled(self) -> bool:
shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
if shutdown:
return False
else:
return True
def type(self) -> str:
if "Ethernet" in self.yy.key:
return "iana-if-type:ethernetCsmacd"
elif "Loopback" in self.yy.key:
return "iana-if-type:softwareLoopback"
else:
raise ValueError(f"don't know type for interface {self.yy.key}")
def loopback_mode(self) -> bool:
if "Loopback" in self.yy.key:
return True
return False
class Interface(Parser):
"""
Implements openconfig-interfaces:interfaces/interface
"""
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text" or "." in interface:
continue
yield interface, data
config = InterfaceConfig
def name(self) -> str:
return self.yy.key
class Interfaces(Parser):
"""
Implements openconfig-interfaces:interfaces
"""
interface = Interface
class LldpConfigState(Parser):
def system_name(self):
return self.yy.native["show run"]["hostname"]["#text"]
class LLdp(Parser):
config = LldpConfigState
Using the new parser in a script to create the lldp container:
[19]:
import task4_lldp_parser
class IOSParser(parser.RootParser):
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = {}
self.root_native["show run"] = parse_indented_config(
self.native["show run"].splitlines()
)
self.root_native["lldp_data"] = self.native["lldp_data"]
self.native = self.root_native
interfaces = task4_lldp_parser.Interfaces
lldp = task4_lldp_parser.LLdp
Execute the script to generate JSON data for the lldp->config containers with a leaf of system-name.
[20]:
p = IOSParser(dm, native=data)
result = p.process(validate=True)
print(json.dumps(result.raw_value(), indent=4))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE1",
"enabled": false
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE2",
"enabled": true
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE3",
"enabled": true
}
},
{
"name": "GigabitEthernet4",
"config": {
"name": "GigabitEthernet4",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE4",
"enabled": true
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE5",
"enabled": true
}
},
{
"name": "Loopback100",
"config": {
"name": "Loopback100",
"type": "iana-if-type:softwareLoopback",
"loopback-mode": true,
"description": "this is loopy100",
"enabled": true
}
}
]
},
"openconfig-lldp:lldp": {
"config": {
"system-name": "ntc-r1"
}
}
}
We are going to add the interfaces container but just return the name of each interface to get started.
Because interfaces is a list, this means we have to implement extract_elements. It’s the same approach we used earlier when parsing interfaces. The only difference is we have a little more logic in extract_elements now:
[21]:
%cat task5_lldp_parser.py
from typing import Any, Dict, Iterator, Optional, Tuple, cast
from yangify.parser import Parser, ParserData
class InterfaceConfig(Parser):
"""
Implements openconfig-interfaces:interfaces/interface/config
"""
def description(self) -> Optional[str]:
return cast(Optional[str], self.yy.native.get("description", {}).get("#text"))
def enabled(self) -> bool:
shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
if shutdown:
return False
else:
return True
def name(self) -> str:
return self.yy.key
def type(self) -> str:
if "Ethernet" in self.yy.key:
return "iana-if-type:ethernetCsmacd"
elif "Loopback" in self.yy.key:
return "iana-if-type:softwareLoopback"
else:
raise ValueError(f"don't know type for interface {self.yy.key}")
def loopback_mode(self) -> bool:
if "Loopback" in self.yy.key:
return True
return False
class Interface(Parser):
"""
Implements openconfig-interfaces:interfaces/interface
"""
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text" or "." in interface:
continue
yield interface, data
config = InterfaceConfig
def name(self) -> str:
return self.yy.key
class Interfaces(Parser):
"""
Implements openconfig-interfaces:interfaces
"""
interface = Interface
class LLDPInterface(Parser):
class Yangify(ParserData):
# While we have LLDP structured data, that is only active
# neighbors found from the show lldp neighbors command
# This LLDP interface is defined as the following within the model
# list interface {
# key "name";
# description
# "List of interfaces on which LLDP is enabled / available";
#
# Because of this, we're using the show run command to build out
# the LLDP interfaces accessed in self.native['show run'] again.
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text":
continue
no_lldp_config = (
data.get("no", {})
.get("lldp", {})
.get("enable", {})
.get("#standalone")
)
if no_lldp_config:
continue
# skip any interfaces you want to...
if interface.startswith("Loop"):
continue
# `interface` is the interface name like GigabitEthernet1
# `data` is the value of the dictionary from the text tree
yield interface, data
def name(self) -> str:
# self.yy.key is equal to the key being returned
# from the tuple in extract_elements
return self.yy.key
class LLDPInterfaces(Parser):
interface = LLDPInterface
class LldpConfigState(Parser):
def system_name(self):
return self.yy.native["show run"]["hostname"]["#text"]
class LLdp(Parser):
config = LldpConfigState
interfaces = LLDPInterfaces
As stated in the docstring, we’re using the show run data to build out the proper framework for the interfaces on the device in contrast to lldp_data that has interfaces that only have active neighbors.
[22]:
import task5_lldp_parser
class IOSParser(parser.RootParser):
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = {}
self.root_native["show run"] = parse_indented_config(
self.native["show run"].splitlines()
)
self.root_native["lldp_data"] = self.native["lldp_data"]
self.native = self.root_native
interfaces = task5_lldp_parser.Interfaces
lldp = task5_lldp_parser.LLdp
[23]:
p = IOSParser(dm, native=data)
result = p.process(validate=False)
print(json.dumps(result.raw_value(), indent=4))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE1",
"enabled": false
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE2",
"enabled": true
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE3",
"enabled": true
}
},
{
"name": "GigabitEthernet4",
"config": {
"name": "GigabitEthernet4",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE4",
"enabled": true
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE5",
"enabled": true
}
},
{
"name": "Loopback100",
"config": {
"name": "Loopback100",
"type": "iana-if-type:softwareLoopback",
"loopback-mode": true,
"description": "this is loopy100",
"enabled": true
}
}
]
},
"openconfig-lldp:lldp": {
"config": {
"system-name": "ntc-r1"
},
"interfaces": {
"interface": [
{
"name": "GigabitEthernet1"
},
{
"name": "GigabitEthernet2"
},
{
"name": "GigabitEthernet3"
},
{
"name": "GigabitEthernet5"
}
]
}
}
}
Just like before, if you do want to see it with validation on, it’ll fail because a name leaf is required in a container called config per interface.
[24]:
from yangson.exceptions import SemanticError
p = IOSParser(dm, native=data)
try:
result = p.process(validate=True)
print(json.dumps(result.raw_value(), indent=4))
except SemanticError as e:
print("Supressing output for readability:")
print(f"error: {e}")
Supressing output for readability:
error: [/openconfig-lldp:lldp/interfaces/interface/0/name] instance-required
Let’s update the parser with the config container that has a single leaf called name using the task6_lldp_parser:
[25]:
%cat task6_lldp_parser.py
from typing import Any, Dict, Iterator, Optional, Tuple, cast
from yangify.parser import Parser, ParserData
class InterfaceConfig(Parser):
"""
Implements openconfig-interfaces:interfaces/interface/config
"""
def description(self) -> Optional[str]:
return cast(Optional[str], self.yy.native.get("description", {}).get("#text"))
def enabled(self) -> bool:
shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
if shutdown:
return False
else:
return True
def name(self) -> str:
return self.yy.key
def type(self) -> str:
if "Ethernet" in self.yy.key:
return "iana-if-type:ethernetCsmacd"
elif "Loopback" in self.yy.key:
return "iana-if-type:softwareLoopback"
else:
raise ValueError(f"don't know type for interface {self.yy.key}")
def loopback_mode(self) -> bool:
if "Loopback" in self.yy.key:
return True
return False
class Interface(Parser):
"""
Implements openconfig-interfaces:interfaces/interface
"""
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text" or "." in interface:
continue
yield interface, data
config = InterfaceConfig
def name(self) -> str:
return self.yy.key
class Interfaces(Parser):
"""
Implements openconfig-interfaces:interfaces
"""
interface = Interface
class LLDPInterfaceInfo(Parser):
def name(self) -> str:
return self.yy.key
class LLDPInterface(Parser):
class Yangify(ParserData):
# While we have LLDP structured data, that is only active
# neighbors found from the show lldp neighbors command
# This LLDP interface is defined as the following within the model
# list interface {
# key "name";
# description
# "List of interfaces on which LLDP is enabled / available";
#
# Because of this, we're using the show run command to build out
# the LLDP interfaces accessed in self.native['show run'] again.
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text":
continue
no_lldp_config = (
data.get("no", {})
.get("lldp", {})
.get("enable", {})
.get("#standalone")
)
if no_lldp_config:
continue
# skip any interfaces you want to...
if interface.startswith("Loop"):
continue
# `interface` is the interface name like GigabitEthernet1
# `data` is the value of the dictionary from the text tree
yield interface, data
config = LLDPInterfaceInfo
def name(self) -> str:
# self.yy.key is equal to the key being returned
# from the tuple in extract_elements
return self.yy.key
class LLDPInterfaces(Parser):
interface = LLDPInterface
class LldpConfigState(Parser):
def system_name(self):
return self.yy.native["show run"]["hostname"]["#text"]
class LLdp(Parser):
config = LldpConfigState
interfaces = LLDPInterfaces
[26]:
import task6_lldp_parser
class IOSParser(parser.RootParser):
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = {}
self.root_native["show run"] = parse_indented_config(
self.native["show run"].splitlines()
)
self.root_native["lldp_data"] = self.native["lldp_data"]
self.native = self.root_native
interfaces = task6_lldp_parser.Interfaces
lldp = task6_lldp_parser.LLdp
[27]:
p = IOSParser(dm, native=data)
aresult = p.process(validate=True)
print(json.dumps(aresult.raw_value(), indent=4))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE1",
"enabled": false
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE2",
"enabled": true
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE3",
"enabled": true
}
},
{
"name": "GigabitEthernet4",
"config": {
"name": "GigabitEthernet4",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE4",
"enabled": true
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE5",
"enabled": true
}
},
{
"name": "Loopback100",
"config": {
"name": "Loopback100",
"type": "iana-if-type:softwareLoopback",
"loopback-mode": true,
"description": "this is loopy100",
"enabled": true
}
}
]
},
"openconfig-lldp:lldp": {
"config": {
"system-name": "ntc-r1"
},
"interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1"
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2"
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3"
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5"
}
}
]
}
}
}
Next, we need to add the neighbors container for each interface, thus this new container, will be at the same hierarchy as config is now. Let’s look at it in task7_lldp_parser:
[28]:
%cat task7_lldp_parser.py
from typing import Any, Dict, Iterator, Optional, Tuple, cast
from yangify.parser import Parser, ParserData
class InterfaceConfig(Parser):
"""
Implements openconfig-interfaces:interfaces/interface/config
"""
def description(self) -> Optional[str]:
return cast(Optional[str], self.yy.native.get("description", {}).get("#text"))
def enabled(self) -> bool:
shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
if shutdown:
return False
else:
return True
def name(self) -> str:
return self.yy.key
def type(self) -> str:
if "Ethernet" in self.yy.key:
return "iana-if-type:ethernetCsmacd"
elif "Loopback" in self.yy.key:
return "iana-if-type:softwareLoopback"
else:
raise ValueError(f"don't know type for interface {self.yy.key}")
def loopback_mode(self) -> bool:
if "Loopback" in self.yy.key:
return True
return False
class Interface(Parser):
"""
Implements openconfig-interfaces:interfaces/interface
"""
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text" or "." in interface:
continue
yield interface, data
config = InterfaceConfig
def name(self) -> str:
return self.yy.key
class Interfaces(Parser):
"""
Implements openconfig-interfaces:interfaces
"""
interface = Interface
class Neighbor(Parser):
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
# take note that self.keys is used to figure out which interface
# is currently in scope
# per the docstring, the self.keys keeps track of all the keys relevant to the current
# object being processed. Keys are equal to the path where the key was extracted while
# the value is the actual value, in our case the value will be the interface name
# we need this because we need to use the interfaces that have LLDP available to them
# to cross-reference it within the second data object that contains
# the actual active neighbors
interface = self.keys["/openconfig-lldp:lldp/interfaces/interface"]
active_neighbors = self.root_native["lldp_data"].get(interface)
if active_neighbors:
for neighbor in active_neighbors:
yield neighbor["neighbor"], neighbor
def id(self) -> str:
return self.yy.key
class Neighbors(Parser):
neighbor = Neighbor
class LLDPInterfaceInfo(Parser):
def name(self) -> str:
return self.yy.key
class LLDPInterface(Parser):
class Yangify(ParserData):
# While we have LLDP structured data, that is only active
# neighbors found from the show lldp neighbors command
# This LLDP interface is defined as the following within the model
# list interface {
# key "name";
# description
# "List of interfaces on which LLDP is enabled / available";
#
# Because of this, we're using the show run command to build out
# the LLDP interfaces accessed in self.native['show run'] again.
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text":
continue
no_lldp_config = (
data.get("no", {})
.get("lldp", {})
.get("enable", {})
.get("#standalone")
)
if no_lldp_config:
continue
# skip any interfaces you want to...
if interface.startswith("Loop"):
continue
# `interface` is the interface name like GigabitEthernet1
# `data` is the value of the dictionary from the text tree
yield interface, data
config = LLDPInterfaceInfo
neighbors = Neighbors
def name(self) -> str:
# self.yy.key is equal to the key being returned
# from the tuple in extract_elements
return self.yy.key
class LLDPInterfaces(Parser):
interface = LLDPInterface
class LldpConfigState(Parser):
def system_name(self):
return self.yy.native["show run"]["hostname"]["#text"]
class LLdp(Parser):
config = LldpConfigState
interfaces = LLDPInterfaces
Pay extra attention to the notes in the docstring this time around because it’s the first we’re showing how to use self.keys. In the current loop (extract_elements), the interfaces from the show run are being iterated over; now we need to take that key, and use it to access the neighbors for that interfaces stored in lldp_data. As you can see, self.keys is Yangify dictionary that helps maintain context on the current object being processed. To see this better, you can explore
it using pdb as mentioned earlier.
[29]:
import json
from yangify import parser
from yangify.parser.text_tree import parse_indented_config
import textfsm
import task7_lldp_parser
class IOSParser(parser.RootParser):
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = {}
self.root_native["show run"] = parse_indented_config(
self.native["show run"].splitlines()
)
self.root_native["lldp_data"] = self.native["lldp_data"]
self.native = self.root_native
interfaces = task7_lldp_parser.Interfaces
lldp = task7_lldp_parser.LLdp
[30]:
p = IOSParser(dm, native=data)
result = p.process(validate=False)
print(json.dumps(result.raw_value(), indent=4))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE1",
"enabled": false
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE2",
"enabled": true
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE3",
"enabled": true
}
},
{
"name": "GigabitEthernet4",
"config": {
"name": "GigabitEthernet4",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE4",
"enabled": true
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE5",
"enabled": true
}
},
{
"name": "Loopback100",
"config": {
"name": "Loopback100",
"type": "iana-if-type:softwareLoopback",
"loopback-mode": true,
"description": "this is loopy100",
"enabled": true
}
}
]
},
"openconfig-lldp:lldp": {
"config": {
"system-name": "ntc-r1"
},
"interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1"
},
"neighbors": {
"neighbor": [
{
"id": "ntc-r6"
},
{
"id": "ntc-r7"
},
{
"id": "ntc-r8"
}
]
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2"
},
"neighbors": {
"neighbor": [
{
"id": "ntc-r2"
},
{
"id": "ntc-s2"
}
]
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3"
},
"neighbors": {
"neighbor": [
{
"id": "ntc-r3"
},
{
"id": "ntc-r5"
}
]
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5"
}
}
]
}
}
}
What’s still missing is the port-id of each neighbor. Let’s add that in (within the state container):
[31]:
%cat task8_lldp_parser.py
from typing import Any, Dict, Iterator, Optional, Tuple, cast
from yangify.parser import Parser, ParserData
class InterfaceConfig(Parser):
"""
Implements openconfig-interfaces:interfaces/interface/config
"""
def description(self) -> Optional[str]:
return cast(Optional[str], self.yy.native.get("description", {}).get("#text"))
def enabled(self) -> bool:
shutdown = self.yy.native.get("shutdown", {}).get("#standalone")
if shutdown:
return False
else:
return True
def name(self) -> str:
return self.yy.key
def type(self) -> str:
if "Ethernet" in self.yy.key:
return "iana-if-type:ethernetCsmacd"
elif "Loopback" in self.yy.key:
return "iana-if-type:softwareLoopback"
else:
raise ValueError(f"don't know type for interface {self.yy.key}")
def loopback_mode(self) -> bool:
if "Loopback" in self.yy.key:
return True
return False
class Interface(Parser):
"""
Implements openconfig-interfaces:interfaces/interface
"""
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text" or "." in interface:
continue
yield interface, data
config = InterfaceConfig
def name(self) -> str:
return self.yy.key
class Interfaces(Parser):
"""
Implements openconfig-interfaces:interfaces
"""
interface = Interface
class State(Parser):
def id(self) -> str:
return self.yy.key
def port_id(self) -> str:
# we already saw self.yy.key is equalivalent to
# what's yielded in extract elements
# now we can see self.yy.native is actually
# equal to the value of what's yielded
return self.yy.native["neighbor_interface"]
class Neighbor(Parser):
class Yangify(ParserData):
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
# take note that self.keys is used to figure out which interface
# is currently in scope
# per the docstring, the self.keys keeps track of all the keys relevant to the current
# object being processed. Keys are equal to the path where the key was extracted while
# the value is the actual value, in our case the value will be the interface name
# we need this because we need to use the interfaces that have LLDP available to them
# to cross-reference it within the second data object that contains
# the actual active neighbors
interface = self.keys["/openconfig-lldp:lldp/interfaces/interface"]
active_neighbors = self.root_native["lldp_data"].get(interface)
if active_neighbors:
for neighbor in active_neighbors:
yield neighbor["neighbor"], neighbor
state = State
def id(self) -> str:
return self.yy.key
class Neighbors(Parser):
neighbor = Neighbor
class LLDPInterfaceInfo(Parser):
def name(self) -> str:
return self.yy.key
class LLDPInterface(Parser):
class Yangify(ParserData):
# While we have LLDP structured data, that is only active
# neighbors found from the show lldp neighbors command
# This LLDP interface is defined as the following within the model
# list interface {
# key "name";
# description
# "List of interfaces on which LLDP is enabled / available";
#
# Because of this, we're using the show run command to build out
# the LLDP interfaces accessed in self.native['show run'] again.
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for interface, data in self.native["show run"]["interface"].items():
if interface == "#text":
continue
no_lldp_config = (
data.get("no", {})
.get("lldp", {})
.get("enable", {})
.get("#standalone")
)
if no_lldp_config:
continue
# skip any interfaces you want to...
if interface.startswith("Loop"):
continue
# `interface` is the interface name like GigabitEthernet1
# `data` is the value of the dictionary from the text tree
yield interface, data
config = LLDPInterfaceInfo
neighbors = Neighbors
def name(self) -> str:
# self.yy.key is equal to the key being returned
# from the tuple in extract_elements
return self.yy.key
class LLDPInterfaces(Parser):
interface = LLDPInterface
class LldpConfigState(Parser):
def system_name(self):
return self.yy.native["show run"]["hostname"]["#text"]
class LLdp(Parser):
config = LldpConfigState
interfaces = LLDPInterfaces
[32]:
import json
from yangify import parser
from yangify.parser.text_tree import parse_indented_config
import textfsm
import task8_lldp_parser
class IOSParser(parser.RootParser):
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = {}
self.root_native["show run"] = parse_indented_config(
self.native["show run"].splitlines()
)
self.root_native["lldp_data"] = self.native["lldp_data"]
self.native = self.root_native
interfaces = task8_lldp_parser.Interfaces
lldp = task8_lldp_parser.LLdp
[33]:
p = IOSParser(dm, native=data, state=True)
result = p.process(validate=True)
print(json.dumps(result.raw_value(), indent=4))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE1",
"enabled": false
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE2",
"enabled": true
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE3",
"enabled": true
}
},
{
"name": "GigabitEthernet4",
"config": {
"name": "GigabitEthernet4",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE4",
"enabled": true
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5",
"type": "iana-if-type:ethernetCsmacd",
"loopback-mode": false,
"description": "Hello GigE5",
"enabled": true
}
},
{
"name": "Loopback100",
"config": {
"name": "Loopback100",
"type": "iana-if-type:softwareLoopback",
"loopback-mode": true,
"description": "this is loopy100",
"enabled": true
}
}
]
},
"openconfig-lldp:lldp": {
"config": {
"system-name": "ntc-r1"
},
"interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"config": {
"name": "GigabitEthernet1"
},
"neighbors": {
"neighbor": [
{
"id": "ntc-r6",
"state": {
"id": "ntc-r6",
"port-id": "Gi0/4"
}
},
{
"id": "ntc-r7",
"state": {
"id": "ntc-r7",
"port-id": "Gi0/2"
}
},
{
"id": "ntc-r8",
"state": {
"id": "ntc-r8",
"port-id": "Gi0/3"
}
}
]
}
},
{
"name": "GigabitEthernet2",
"config": {
"name": "GigabitEthernet2"
},
"neighbors": {
"neighbor": [
{
"id": "ntc-r2",
"state": {
"id": "ntc-r2",
"port-id": "Gi0/1"
}
},
{
"id": "ntc-s2",
"state": {
"id": "ntc-s2",
"port-id": "Gi0/1"
}
}
]
}
},
{
"name": "GigabitEthernet3",
"config": {
"name": "GigabitEthernet3"
},
"neighbors": {
"neighbor": [
{
"id": "ntc-r3",
"state": {
"id": "ntc-r3",
"port-id": "Gi0/3"
}
},
{
"id": "ntc-r5",
"state": {
"id": "ntc-r5",
"port-id": "Gi0/4"
}
}
]
}
},
{
"name": "GigabitEthernet5",
"config": {
"name": "GigabitEthernet5"
}
}
]
}
}
}
Hopefully this helps really convey what Yangify is all about, how it can be used, and gets you on your way for buliding Yangify parsers!