Before we get into the topic, let’s get some definitions right first:

  1. Iterator – this is an object that can be iterated upon. An iterator object returns data, one element at a time. Some of python’s containers that are iterable include lists, tuples, strings. These have the __iter__ method defined, which when called returns an iterator object.
  2. Generator – a simple way of creating iterators defined above.
  3. Coroutine – same concept as generators but we’ll see how they differ later in the article.

To better illustrate, consider a list:

 my_list = [1, 2, 3, 4, 5]
  iter_obj = iter(my_list) # calls __iter__ , and an iterable object is returned
  next(iter_obj) # prints 1
  next(iter_obj) # prints 2

And so on, till we have gotten all items from the list after which the StopIteration exception is raised. By the way, we can also call iter_obj.__next__()

In order to build your own iterators, you have to implement a class with the __iter__ and __next__ methods. At times this can be very cumbersome, and that is where generators come in. Generators allow us to easily create iterators. All implementations mentioned above are not required as they are automatically handled. It’s fairly simple to create a generator in python: as easy as defining a normal function with the yield statement rather than the return statement. If you’re wondering what’s the difference between these two terms; return terminates a function while yield pauses the function while saving its state and will continue from where it left off on successive calls.

As an example, we can define a simple function that is a generator for Fibonacci numbers. (Some people might argue that the sequence starts from 0 but let’s focus on the python here :

 def fibonacci(limit):
   n2 = 1
   if limit >= 1:
       yield n2

   n1 = 0

   for _ in range(1, limit):
       n = n1 + n2
       yield n
       n1, n2 = n2, n

To get the data, we can do:

Fib = fibonacci(5) # we limit the number of elements we get to 5
next(fib) # prints 1
next(fib) # prints 1
next(fib) # prints 2
next(fib) # prints 3
next(fib) # prints 5
next(fib) # StopIteration exception is raised

Simple, isn’t it? On to coroutines now. Please note that in this article I talk about simple coroutines rather than native coroutines.

We’ve seen that generators allow us to pull data and pause execution from a function context. Coroutines allow us to push data. In this case, the yield statement basically means “Wait until you get some input data”. We also use the yield statement a bit different as we shall see in a bit. 

We’ll go through a very simple example which provides a good basis: Imagine we have a list of names and want to find out if some names we have in mind (we don’t know how many since they’ll be input at runtime) are in this list.

def check_name_exists():
   names = ["Dennis", "Nick", "Fury", "Tony", "Stark"]
   print("Ready to check for names.")
   while True:
       name = yield
       if name in names:
           print("Found")
       else:
           print("Not found")

We can now do:

coro = check_name_exists() 

However, a coroutine can’t start receiving data right away and we’ll get an error when we try to do so. We first need to prime it. This can be done easily by doing:

 coro.send(None) # prints out Ready to check for names.

Now, we can start sending some data to our coroutine, and this is done by making use of the send method.

coro.send("Dennis") # prints out Found
coro.send("Captain") # prints out Not found

An important thing to note is if our generator or coroutine does not have a breakpoint, we can manually stop it by doing:

coro.close()

That’s it for now, I hope this was helpful in beginning to understand generators and coroutines. You can read more from these resources:

READ: How to build your own version of React from scratch (Part 1)

featured_image
About the Author

Dennis Wainaina

Senior Software Engineer at Andela

More Insights

September 30, 2019

An Introduction to Python Generators and Coroutines

Dennis Wainaina

Before we get into the topic, let’s get some definitions right first:

  1. Iterator – this is an object that can be iterated upon. An iterator object returns data, one element at a time. Some of python’s containers that are iterable include lists, tuples, strings. These have the __iter__ method defined, which when called returns an iterator object.
  2. Generator – a simple way of creating iterators defined above.
  3. Coroutine – same concept as generators but we’ll see how they differ later in the article.

To better illustrate, consider a list:

 my_list = [1, 2, 3, 4, 5]
  iter_obj = iter(my_list) # calls __iter__ , and an iterable object is returned
  next(iter_obj) # prints 1
  next(iter_obj) # prints 2

And so on, till we have gotten all items from the list after which the StopIteration exception is raised. By the way, we can also call iter_obj.__next__()

In order to build your own iterators, you have to implement a class with the __iter__ and __next__ methods. At times this can be very cumbersome, and that is where generators come in. Generators allow us to easily create iterators. All implementations mentioned above are not required as they are automatically handled. It’s fairly simple to create a generator in python: as easy as defining a normal function with the yield statement rather than the return statement. If you’re wondering what’s the difference between these two terms; return terminates a function while yield pauses the function while saving its state and will continue from where it left off on successive calls.

As an example, we can define a simple function that is a generator for Fibonacci numbers. (Some people might argue that the sequence starts from 0 but let’s focus on the python here :

 def fibonacci(limit):
   n2 = 1
   if limit >= 1:
       yield n2

   n1 = 0

   for _ in range(1, limit):
       n = n1 + n2
       yield n
       n1, n2 = n2, n

To get the data, we can do:

Fib = fibonacci(5) # we limit the number of elements we get to 5
next(fib) # prints 1
next(fib) # prints 1
next(fib) # prints 2
next(fib) # prints 3
next(fib) # prints 5
next(fib) # StopIteration exception is raised

Simple, isn’t it? On to coroutines now. Please note that in this article I talk about simple coroutines rather than native coroutines.

We’ve seen that generators allow us to pull data and pause execution from a function context. Coroutines allow us to push data. In this case, the yield statement basically means “Wait until you get some input data”. We also use the yield statement a bit different as we shall see in a bit. 

We’ll go through a very simple example which provides a good basis: Imagine we have a list of names and want to find out if some names we have in mind (we don’t know how many since they’ll be input at runtime) are in this list.

def check_name_exists():
   names = ["Dennis", "Nick", "Fury", "Tony", "Stark"]
   print("Ready to check for names.")
   while True:
       name = yield
       if name in names:
           print("Found")
       else:
           print("Not found")

We can now do:

coro = check_name_exists() 

However, a coroutine can’t start receiving data right away and we’ll get an error when we try to do so. We first need to prime it. This can be done easily by doing:

 coro.send(None) # prints out Ready to check for names.

Now, we can start sending some data to our coroutine, and this is done by making use of the send method.

coro.send("Dennis") # prints out Found
coro.send("Captain") # prints out Not found

An important thing to note is if our generator or coroutine does not have a breakpoint, we can manually stop it by doing:

coro.close()

That’s it for now, I hope this was helpful in beginning to understand generators and coroutines. You can read more from these resources:

READ: How to build your own version of React from scratch (Part 1)

featured_image
About the Author

Dennis Wainaina

Senior Software Engineer at Andela

Thanks for subscribing!

 

More Insights

Partners in Delivery: Andela’s Model of Remote Software Engineering Staff Augmentation

In the wake of the COVID-19 pandemic, businesses are looking to streamline operations and find new ...

4_June_2020

How to keep engaging your tech community in a fully-remote world

In-person (offline) tech community meetups have, for the longest time, been the major engagement pl...

29_May_2020

The complete guide to Debug Swift code with LLDB

This guide contains the following content to ease your journey to become an lldb ninja: — Obje...

27_May_2020

Tips for Handling Remote Team Emergencies

Guest post by Ashley Kent. If your company is in the process of transitioning to a remote team, ...

27_May_2020

Remote Engineering Staff Augmentation Aids the Race to Digitize

While the economic shock and aftershocks of the global pandemic continue to ripple through the econ...

26_May_2020

Partners

Tap into a global talent pool and hire the “right” developers in days, not months.

Developers

Accelerate your career by working with high-performing engineering teams around the world.

BECOME A DEVELOPER

Hire Developers

We take great pride in matching our developers with the best partners. Tell us about your team below!

preloader_image

Thank you for your interest

A member of our team will reach out to you soon.