Geek StuffPython

Python String, List and Dictionary Templating

python

Introduction

Have you tried automating any networking service ? During automation (especially using legacy APIs of networking hardware vendors), you may encounter a lot of boilerplate configuration data. Using templating, we can easily abstract such data in template. If you know Django, you might already be familiar with templating. In Python ecosystem, many templating engines are available. But Jinja2 is the most popular. Most of these engines accept template written only in string. But sometimes, it can be very convenient to have template in dictionary or list. So we can directly convert it in JSON and use in HTTP body. In this article, we will write a small code snippet to generate template using python datatypes (like list, tuple, dictionary or string). First let’s see what are other features available in Python to manipulate strings with dynamic data.

Python String Interpolation

Python has the interesting features for string interpolation. See below examples.

Example 1

string1 = "Hello I am %(name)s, from %(location)s. My blog is at %(blog)s."
values = {"name": "Rohit", "location": "Earth", "blog": "LaymanClass"}
print(string1 %values)

Output

Hello I am Rohit, from Earth. My blog is at LaymanClass.

Example 2

string2 = "Hello I am {name} from {location}. My blog is at {blog}."
values = {"name": "Rohit", "location": "Earth", "blog": "LaymanClass"}
print(string2.format(**values))

Output

Hello I am Rohit from Earth. My blog is at LaymanClass.

We need a similar feature but instead of string our template will be in either dictionary or list.

Python 3.6 f-string

With launch Python3.6, f-strings are introduced. Using f-string, we can solve our problem easily. See below example.

value3 = "foo"
child2 = "bar"
child_value2 = "baz"
TEMPLATE = {
    "object": {
        "attributes": {
          "property1": "value1",
          "property2": "value2",
          "property3": f"{value3}"
        },

        "children": [
            {
                "%(child1)s": {
                    "attributes": {
                        "child1_property1": "child_value1",
                        "child1_property2": "child_value2",
                        "child1_property3": f"{child_value2}",
                    },
                },

                f"{child2}": {
                    "attributes": {
                        "child2_property1": "child_value1",
                        "child2_property2": "child_value2",
                        "child2_property3": f"{child_value2}",
                    },
                },
            },
          ]

        }
}

But as you have seen, for this to work, we have to define all our variables before. This makes abstracting templates in different module difficult.

Solution

So our final solution is – write a helper function which will take python objects as template and will work similar to string interpolation. See below snippet.

# Ref - https://github.com/rohitchormale/cheats-et-scripts/blob/master/python/data_template.py

import re

def fill_template(template, values, key_lookup=True, lookup_regex="%\((.*?)\)s"):
    """Fill complex dictionary templates with values from another dict.
  
    Parameters:
    template (dict): Dictionary template with placeholders to fill up
    values (dict): Dictionary of placeholder-values. Each placeholder in template will be replaced by related value in this dictionary 
    key_lookup (boolean): By default, both key and values having placeholders will be filled up. To disable, key fill up, set this option 'False'
    lookup_regex (string): Provide custom regex for placeholder. Default placeholder pattern will be '%(placeholder)s'
    Returns:
    dict: Template by filling up values
    """

    def lookup(text):
        try:
            placeholder = re.compile(lookup_regex).search(text).group(1)
        except AttributeError:
            placeholder = None
        if placeholder is not None and placeholder in values:
            return values[placeholder]
        return text

    def fill_list(list_template):
        temp = []
        for i in list_template:
            if isinstance(i, str) and i.strip() != "":
                temp.append(lookup(i))
            elif isinstance(i, dict):
                filled_dict = fill_dict(i)
                temp.append(filled_dict)
            elif isinstance(i, list):
                filled_list = fill_list(i)
                temp.append(filled_list)
            else:
                temp.append(i)
        return temp


    def fill_dict(dict_template):
        temp = {}
        for i, j in dict_template.items():
            k = lookup(i) if key_lookup else i
            if isinstance(j, str) and j.strip() != "":
                temp[k] = lookup(j)
            elif isinstance(j, dict):
                filled_dict = fill_dict(j)
                temp[k] = filled_dict
            elif isinstance(j, list):
                filled_list = fill_list(j) 
                temp[k] = filled_list
            else:
                temp[k] = j
        return temp

    if isinstance(template, list):
        return fill_list(template)
    elif isinstance(template, tuple):
        output = fill_list(template)
        return tuple(output)
    elif isinstance(template, dict):
        return fill_dict(template)
    else:
        return lookup(template)

Let’s test this snippet.

template = {
            "key1": "%(key1)s",
            "key2": "%(key2)s",
            "key3": ["foo", "bar", {"key4": "%(key1)s", "key5": "%(key1)s"}],
            "key6": {"key7": "%(key7)s", "key8": "%(key8)s", "%(key9)s": ["foo", {"key10": "%(key10)s", "key11": "%(key11)s", "key12": "%(key12)s"}, "bar"]},
            "key13": "%(key13missing)s"
        }

values = {
         "key1": "value1", "key2": "value2", "key3": "value3", "key4": "value4", "key5": "value5", "key6": "value6", "key7": "value7", "key8": "value8", "key9": "value9", "key10": "value10", "key11": "value11", "key12": "value12" }

fill_template(template, values)

Output will be below –

{'key1': 'value1',
 'key2': 'value2',
 'key3': ['foo', 'bar', {'key4': 'value1', 'key5': 'value1'}],
 'key6': {'key7': 'value7',
  'key8': 'value8',
  'value9': ['foo',
   {'key10': 'value10', 'key11': 'value11', 'key12': 'value12'},
   'bar']},
 'key13': '%(key13missing)s'}

This snippet also support custom placeholder. When defining template, you can modify placeholder pattern easily by providing regex parameter.

Hope you enjoy the article ! Stay awesome !

Leave a Reply