Diff Lists

Identify the differences between two input lists.

Use Case

You need to compare two lists and highlight the unique entries or entries exclusive to the first list, and require a transformation action that simplifies this complex comparison process.

Overview

The Diff Lists transform is your solution for list comparison tasks. It uses two methods - Anti-Join and Symmetric Difference - to focus on unique or exclusive entries, thereby streamlining your data comparison process.

---

Parameters

Here are the parameters you have available to you within the action, and their descriptions:

Parameter	Description	Required
Difference Method	Choose the method to compute the differences between the lists. Options are anti_join and symmetric_difference (described below).
First List	Enter the first list to be compared.
First List's Key	Identify the key attribute for matching items in the first list.
Second List	Enter the second list to be compared.
Second List's Key	Identify the key attribute for matching items in the second list.

Usage

Input Lists

For this transform, we will want to feed the action two separate lists, that we will then identify the fields in which we want to identify differences. Let's use these two lists as example:

List 1:

list_1: [
  {"id": 1, "name": "John"},
  {"id": 2, "name": "Mary"}
]

List 2:

list_2: [
  {"id": 1, "age": 30},
  {"id": 3, "age": 35}
]

Difference Methods

We have two options for the Difference Method you can pick to have the action perform, these method have slight differences that are quite helpful in various situations:

• Anti-Join: for scenarios where you want to find entries that are exclusive to the first list.

• Symmetric Difference: for comparing two sets of data and identifying unique records in each.

Take a look at examples of both below:

The Symmetric Difference method is used when you want to identify entries that are unique to each list, i.e., items that exist in the one list and not in the other.

The Anti Join difference method is used when you want to determine the items in the first list that don't have a corresponding match in the second list.

---

Anti Join

Action Parameters:

diff_method: anti_join
list_1: List 1
list1_key: id
list_2: List 2
list2_key: id

Jinja2 Equivalent:

{% set result = [] %}
{% for item1 in list_1 %}
  {% if all(item1[list1_key] != item2[list2_key] for item2 in list_2) %}
    {% do result.append(item1) %}
  {% endif %}
{% endfor %}


{{ result }}

Symmetric Difference

Action Parameters:

diff_method: symmetric_difference
list_1: {{ CTX.list_1 }}
list1_key: id
list_2: {{ CTX.list_2 }}
list2_key: id

Jinja2 Equivalent:

{% set result = [] %}
{% for item1 in list_1 %}
  {% if all(item1[list1_key] != item2[list2_key] for item2 in list_2) %}
    {% do result.append(item1) %}
  {% endif %}
{% endfor %}
{% for item2 in list_2 %}
  {% if all(item2[list2_key] != item1[list1_key] for item1 in list_1) %}
    {% do result.append(item2) %}
  {% endif %}
{% endfor %}


{{ result }}

Results Output

The results of the two different diff methods can be seen as follows:

Anti Join: only Mary's object from list_1 is returned in the output, as she didn't have a correlating id in list_2.

result: [
  {"id": 2, "name": "Mary"}
]

Symmetric Difference: both Mary's object from list_1 and the object in list_2 that didn't have a correlating id in list_1 were returned in the returned list.

result: [
  {"id": 2, "name": "Mary"},
  {"id": 3, "age": 35}
]

---

With a grasp on the Diff Lists transform, you're ready to analyze and compare your lists for unique entries or entries exclusive to the first list. Understanding the differences between Anti-Join and Symmetric Difference methods will aid you in making effective data comparisons.

Want more practice?

Check out our Rewst Foundations Courses.