Before we get into the topic, let’s get some definitions right first:

  1. Iterator – this is an object that can be iterated upon. An iterator object returns data, one element at a time. Some of python’s containers that are iterable include lists, tuples, strings. These have the __iter__ method defined, which when called returns an iterator object.
  2. Generator – a simple way of creating iterators defined above.
  3. Coroutine – same concept as generators but we’ll see how they differ later in the article.

To better illustrate, consider a list:

 my_list = [1, 2, 3, 4, 5]
  iter_obj = iter(my_list) # calls __iter__ , and an iterable object is returned
  next(iter_obj) # prints 1
  next(iter_obj) # prints 2

And so on, till we have gotten all items from the list after which the StopIteration exception is raised. By the way, we can also call iter_obj.__next__()

In order to build your own iterators, you have to implement a class with the __iter__ and __next__ methods. At times this can be very cumbersome, and that is where generators come in. Generators allow us to easily create iterators. All implementations mentioned above are not required as they are automatically handled. It’s fairly simple to create a generator in python: as easy as defining a normal function with the yield statement rather than the return statement. If you’re wondering what’s the difference between these two terms; return terminates a function while yield pauses the function while saving its state and will continue from where it left off on successive calls.

As an example, we can define a simple function that is a generator for Fibonacci numbers. (Some people might argue that the sequence starts from 0 but let’s focus on the python here :

 def fibonacci(limit):
   n2 = 1
   if limit >= 1:
       yield n2

   n1 = 0

   for _ in range(1, limit):
       n = n1 + n2
       yield n
       n1, n2 = n2, n

To get the data, we can do:

Fib = fibonacci(5) # we limit the number of elements we get to 5
next(fib) # prints 1
next(fib) # prints 1
next(fib) # prints 2
next(fib) # prints 3
next(fib) # prints 5
next(fib) # StopIteration exception is raised

Simple, isn’t it? On to coroutines now. Please note that in this article I talk about simple coroutines rather than native coroutines.

We’ve seen that generators allow us to pull data and pause execution from a function context. Coroutines allow us to push data. In this case, the yield statement basically means “Wait until you get some input data”. We also use the yield statement a bit different as we shall see in a bit. 

We’ll go through a very simple example which provides a good basis: Imagine we have a list of names and want to find out if some names we have in mind (we don’t know how many since they’ll be input at runtime) are in this list.

def check_name_exists():
   names = ["Dennis", "Nick", "Fury", "Tony", "Stark"]
   print("Ready to check for names.")
   while True:
       name = yield
       if name in names:
           print("Found")
       else:
           print("Not found")

We can now do:

coro = check_name_exists() 

However, a coroutine can’t start receiving data right away and we’ll get an error when we try to do so. We first need to prime it. This can be done easily by doing:

 coro.send(None) # prints out Ready to check for names.

Now, we can start sending some data to our coroutine, and this is done by making use of the send method.

coro.send("Dennis") # prints out Found
coro.send("Captain") # prints out Not found

An important thing to note is if our generator or coroutine does not have a breakpoint, we can manually stop it by doing:

coro.close()

That’s it for now, I hope this was helpful in beginning to understand generators and coroutines. You can read more from these resources:

READ: How to build your own version of React from scratch (Part 1)

featured_image
About the Author

Dennis Wainaina

Senior Software Engineer at Andela

Thanks for subscribing!

 

More Insights

September 30, 2019

An Introduction to Python Generators and Coroutines

Dennis Wainaina

Before we get into the topic, let’s get some definitions right first:

  1. Iterator – this is an object that can be iterated upon. An iterator object returns data, one element at a time. Some of python’s containers that are iterable include lists, tuples, strings. These have the __iter__ method defined, which when called returns an iterator object.
  2. Generator – a simple way of creating iterators defined above.
  3. Coroutine – same concept as generators but we’ll see how they differ later in the article.

To better illustrate, consider a list:

 my_list = [1, 2, 3, 4, 5]
  iter_obj = iter(my_list) # calls __iter__ , and an iterable object is returned
  next(iter_obj) # prints 1
  next(iter_obj) # prints 2

And so on, till we have gotten all items from the list after which the StopIteration exception is raised. By the way, we can also call iter_obj.__next__()

In order to build your own iterators, you have to implement a class with the __iter__ and __next__ methods. At times this can be very cumbersome, and that is where generators come in. Generators allow us to easily create iterators. All implementations mentioned above are not required as they are automatically handled. It’s fairly simple to create a generator in python: as easy as defining a normal function with the yield statement rather than the return statement. If you’re wondering what’s the difference between these two terms; return terminates a function while yield pauses the function while saving its state and will continue from where it left off on successive calls.

As an example, we can define a simple function that is a generator for Fibonacci numbers. (Some people might argue that the sequence starts from 0 but let’s focus on the python here :

 def fibonacci(limit):
   n2 = 1
   if limit >= 1:
       yield n2

   n1 = 0

   for _ in range(1, limit):
       n = n1 + n2
       yield n
       n1, n2 = n2, n

To get the data, we can do:

Fib = fibonacci(5) # we limit the number of elements we get to 5
next(fib) # prints 1
next(fib) # prints 1
next(fib) # prints 2
next(fib) # prints 3
next(fib) # prints 5
next(fib) # StopIteration exception is raised

Simple, isn’t it? On to coroutines now. Please note that in this article I talk about simple coroutines rather than native coroutines.

We’ve seen that generators allow us to pull data and pause execution from a function context. Coroutines allow us to push data. In this case, the yield statement basically means “Wait until you get some input data”. We also use the yield statement a bit different as we shall see in a bit. 

We’ll go through a very simple example which provides a good basis: Imagine we have a list of names and want to find out if some names we have in mind (we don’t know how many since they’ll be input at runtime) are in this list.

def check_name_exists():
   names = ["Dennis", "Nick", "Fury", "Tony", "Stark"]
   print("Ready to check for names.")
   while True:
       name = yield
       if name in names:
           print("Found")
       else:
           print("Not found")

We can now do:

coro = check_name_exists() 

However, a coroutine can’t start receiving data right away and we’ll get an error when we try to do so. We first need to prime it. This can be done easily by doing:

 coro.send(None) # prints out Ready to check for names.

Now, we can start sending some data to our coroutine, and this is done by making use of the send method.

coro.send("Dennis") # prints out Found
coro.send("Captain") # prints out Not found

An important thing to note is if our generator or coroutine does not have a breakpoint, we can manually stop it by doing:

coro.close()

That’s it for now, I hope this was helpful in beginning to understand generators and coroutines. You can read more from these resources:

READ: How to build your own version of React from scratch (Part 1)

featured_image
About the Author

Dennis Wainaina

Senior Software Engineer at Andela

Thanks for subscribing!

 

More Insights

Urban Ishimwe’s Software Dev Journey

Perhaps in another life, Urban Ishimwe might have been some famous Social Media Influencer helping b...

16_October_2019

Africa’s Talking Node.js (express) USSD Application

The app developed here will help a user to view their phone number and account details (Account numb...

10_October_2019

Practices and behaviours of highly productive remote teams

I have worked remotely as a Software engineer with several companies for over four years now. This a...

7_October_2019

Andela & GitHub Partner to Host CodeNaija 2019; Nigeria’s Biggest Hackathon Event of the Year

We’re excited to announce that we will be hosting the CodeNaija 2019 hackathon event in partnershi...

4_October_2019

Tapping into your Dev Beast Mode

Before jumping into the gist of this article it is important to understand what it hopes to achieve ...

19_September_2019

The Future of Andela

We started Andela five years ago to solve a simple but pervasive global challenge: Brilliance is eve...

17_September_2019

Partners

Tap into a global talent pool and hire the “right” developers in days, not months.

Developers

Accelerate your career by working with high-performing engineering teams around the world.

BECOME A DEVELOPER

Hire Developers

We take great pride in matching our developers with the best partners. Tell us about your team below!

preloader_image

Thank you for your interest

A member of our team will reach out to you soon.