Introduction
Have you tried automating any networking service ? During automation (especially using legacy APIs of networking hardware vendors), you may encounter a lot of boilerplate configuration data. Using templating, we can easily abstract such data in template. If you know Django, you might already be familiar with templating. In Python ecosystem, many templating engines are available. But Jinja2 is the most popular. Most of these engines accept template written only in string. But sometimes, it can be very convenient to have template in dictionary or list. So we can directly convert it in JSON and use in HTTP body. In this article, we will write a small code snippet to generate template using python datatypes (like list, tuple, dictionary or string). First let’s see what are other features available in Python to manipulate strings with dynamic data.
Python String Interpolation
Python has the interesting features for string interpolation. See below examples.
Example 1
string1 = "Hello I am %(name)s, from %(location)s. My blog is at %(blog)s."
values = {"name": "Rohit", "location": "Earth", "blog": "LaymanClass"}
print(string1 %values)Output
Hello I am Rohit, from Earth. My blog is at LaymanClass.Example 2
string2 = "Hello I am {name} from {location}. My blog is at {blog}."
values = {"name": "Rohit", "location": "Earth", "blog": "LaymanClass"}
print(string2.format(**values))Output
Hello I am Rohit from Earth. My blog is at LaymanClass.We need a similar feature but instead of string our template will be in either dictionary or list.
Python 3.6 f-string
With launch Python3.6, f-strings are introduced. Using f-string, we can solve our problem easily. See below example.
value3 = "foo"
child2 = "bar"
child_value2 = "baz"
TEMPLATE = {
"object": {
"attributes": {
"property1": "value1",
"property2": "value2",
"property3": f"{value3}"
},
"children": [
{
"%(child1)s": {
"attributes": {
"child1_property1": "child_value1",
"child1_property2": "child_value2",
"child1_property3": f"{child_value2}",
},
},
f"{child2}": {
"attributes": {
"child2_property1": "child_value1",
"child2_property2": "child_value2",
"child2_property3": f"{child_value2}",
},
},
},
]
}
}But as you have seen, for this to work, we have to define all our variables before. This makes abstracting templates in different module difficult.
Solution
So our final solution is – write a helper function which will take python objects as template and will work similar to string interpolation. See below snippet.
# Ref - https://github.com/rohitchormale/cheats-et-scripts/blob/master/python/data_template.py
import re
def fill_template(template, values, key_lookup=True, lookup_regex="%\((.*?)\)s"):
"""Fill complex dictionary templates with values from another dict.
Parameters:
template (dict): Dictionary template with placeholders to fill up
values (dict): Dictionary of placeholder-values. Each placeholder in template will be replaced by related value in this dictionary
key_lookup (boolean): By default, both key and values having placeholders will be filled up. To disable, key fill up, set this option 'False'
lookup_regex (string): Provide custom regex for placeholder. Default placeholder pattern will be '%(placeholder)s'
Returns:
dict: Template by filling up values
"""
def lookup(text):
try:
placeholder = re.compile(lookup_regex).search(text).group(1)
except AttributeError:
placeholder = None
if placeholder is not None and placeholder in values:
return values[placeholder]
return text
def fill_list(list_template):
temp = []
for i in list_template:
if isinstance(i, str) and i.strip() != "":
temp.append(lookup(i))
elif isinstance(i, dict):
filled_dict = fill_dict(i)
temp.append(filled_dict)
elif isinstance(i, list):
filled_list = fill_list(i)
temp.append(filled_list)
else:
temp.append(i)
return temp
def fill_dict(dict_template):
temp = {}
for i, j in dict_template.items():
k = lookup(i) if key_lookup else i
if isinstance(j, str) and j.strip() != "":
temp[k] = lookup(j)
elif isinstance(j, dict):
filled_dict = fill_dict(j)
temp[k] = filled_dict
elif isinstance(j, list):
filled_list = fill_list(j)
temp[k] = filled_list
else:
temp[k] = j
return temp
if isinstance(template, list):
return fill_list(template)
elif isinstance(template, tuple):
output = fill_list(template)
return tuple(output)
elif isinstance(template, dict):
return fill_dict(template)
else:
return lookup(template)Let’s test this snippet.
template = {
"key1": "%(key1)s",
"key2": "%(key2)s",
"key3": ["foo", "bar", {"key4": "%(key1)s", "key5": "%(key1)s"}],
"key6": {"key7": "%(key7)s", "key8": "%(key8)s", "%(key9)s": ["foo", {"key10": "%(key10)s", "key11": "%(key11)s", "key12": "%(key12)s"}, "bar"]},
"key13": "%(key13missing)s"
}
values = {
"key1": "value1", "key2": "value2", "key3": "value3", "key4": "value4", "key5": "value5", "key6": "value6", "key7": "value7", "key8": "value8", "key9": "value9", "key10": "value10", "key11": "value11", "key12": "value12" }
fill_template(template, values)
Output will be below –
{'key1': 'value1',
'key2': 'value2',
'key3': ['foo', 'bar', {'key4': 'value1', 'key5': 'value1'}],
'key6': {'key7': 'value7',
'key8': 'value8',
'value9': ['foo',
{'key10': 'value10', 'key11': 'value11', 'key12': 'value12'},
'bar']},
'key13': '%(key13missing)s'}This snippet also support custom placeholder. When defining template, you can modify placeholder pattern easily by providing regex parameter.
Hope you enjoy the article ! Stay awesome !