Flight booking automation presents significant challenges, primarily due to the scarcity of public APIs. Consequently, the process of searching for flights often requires simulating human-like interactions with web interfaces.

Fortunately, the combination of CrewAI and Browserbase only requires a few dozen lines of code to automate this complex task.

By following this tutorial, you’ll learn how to build a CrewAI program that searches for a roundtrip flight from a simple human input:

> python3 main.py "San Francisco to NYC one-way on September 21st"

Here are our top 5 flights from San Francisco (SFO) to Newark (EWR) on September 21, 2024:

1. **Alaska Airlines**:
   - Departure: 8:50 am
   - Arrival: 5:24 pm
   - Duration: 5 hours 34 minutes
   - Layovers: Nonstop
   - Price: $125
   - Booking: [Alaska Airlines Saver](https://www.kayak.com/book/flight?code=noAiOYx8xU.4fFBlTtfVpoDzQq2dWkU9A.12411.28f6c8a3257adb48c2f7d8207660b2a0&h=41a638bca25d&_kw_pbranded=true&sub=F-1450586051791345216E0040d85ce85&pageOrigin=F..RP.FE.M4)
...

​
Introduction: Crews, Agents, Tasks and Tools

CrewAI helps developers build AI Agents with 4 core concepts: Crews, Agents, Tasks, and Tools:

  • A Crew is a team of Agents working together to accomplish some tasks.
  • A Task, such as “Search flights according to criteria”, is a goal assigned to a specialized Agent (e.g., a Flight Booking Agent).
  • An Agent can be seen as a specialized text-only GPT that receives a set of Tools to perform actions (e.g., search on Google, navigate to this URL).

​
Example

Here is an example of a Crew assembled to research a given topic and write an article.

The Agents: A Researcher and a Writer

First, let’s define 2 Agents, one specialized in researching a topic and another in writing articles:

researcher = Agent(
  role='Senior Researcher',
  goal='Uncover groundbreaking technologies in {topic}',
  backstory=(
    "Driven by curiosity, you're at the forefront of"
    "innovation, eager to explore and share knowledge that could change"
    "the world."
  ),
  tools=[search_tool],
)

writer = Agent(
  role='Writer',
  goal='Narrate compelling tech stories about {topic}',
  backstory=(
    "With a flair for simplifying complex topics, you craft"
    "engaging narratives that captivate and educate, bringing new"
    "discoveries to light in an accessible manner."
  ),
  tools=[search_tool]
)

Each Agent gets:

  • a role that helps the Crew select the best Agent for a given Task.
  • a goal that frames the Agent decision-making process when iterating on a Task.
  • a backstory providing context to the Agent’s role and goal.

Both Agents get access to a search_tool (SerperDevTool instance) to perform searches with Google Search.

The Tasks: writing and researching

Let’s now define 2 tasks: researching a topic and writing an article.

research_task = Task(
  description=(
    "Identify the next big trend in {topic}."
    "Focus on identifying pros and cons and the overall narrative."
    "Your final report should clearly articulate the key points,"
    "its market opportunities, and potential risks."
  ),
  expected_output='A comprehensive 3 paragraphs long report on the latest AI trends.',
  agent=researcher,
)

write_task = Task(
  description=(
    "Compose an insightful article on {topic}."
    "Focus on the latest trends and how it's impacting the industry."
    "This article should be easy to understand, engaging, and positive."
  ),
  expected_output='A 4 paragraph article on {topic} advancements formatted as markdown.',
  agent=writer,
  output_file='new-blog-post.md'  # Example of output customization
)

A Task’s description can be compared to a prompt, while the expected_output helps format the result of the Task.

As expected, the write_task gets assigned to the writer Agent and the research_task to the researcher Agent.

Agents and Tasks look very similar: do I need both?

Indeed, in a simple example as this one, the Agent and Task look alike. In real-world applications, an Agent gets to perform multiple tasks. Then, an Agent represents the expertise (goal, backstory) with a set of skills (tools), while a Task is a goal to accomplish.

Assembling the Crew

As covered earlier, a Crew defines a set of Task to be performed sequentially by a team of Agents.

Note that Tasks share a context, explaining why the research task comes before the writing task.

crew = Crew(
  agents=[researcher, writer],
  tasks=[research_task, write_task],
  memory=True,
  cache=True,
  max_rpm=100,
)

result = crew.kickoff(inputs={'topic': 'AI in healthcare'})
print(result)

Let’s now build our Flight Booking Crew with these fresh new concepts!

​
1. Our Flight Booking Crew

Before jumping into the setup and code, let’s step back and look at how to assemble a Crew that helps book flights.

From a user input like “San Francisco to New York one-way on 21st September”, our Flight Booking Crew should print the top 5 flights as follows:

Here are our top 5 picks from San Francisco to New York on 21st September 2024:
1. **Delta Airlines**
   - Departure: 21:35
   - Arrival: 03:50
   - Duration: 6 hours 15 minutes
   - Layovers: Direct
   - Price: $125
   - Booking: [Delta Airlines](https://www.kayak.com/flights/sfo/jfk/2024-09-21/12:45/13:55/2:10/delta/airlines/economy/1)
...

To achieve this goal, our Crew will navigate to https://www.kayak.com, perform a search, and extract each flight detail, which translates to the following steps:

  1. Parse the user request (“San Francisco to New York one-way on 21st September”) to build a valid Kayak search URL
  2. Navigate to the Kayak search URL and extract the top 5 flights
  3. For each flight, navigate to the flight details URL to extract the available providers (airlines)
  4. Summarize the flights’ information

To perform those steps, we will create 2 Agents:

  • The “Flights” Agent, responsible for looking for flights
  • The “Summarize” Agent, responsible for summarizing the available flights as a comprehensive list

The “Search Flights” Agent will need:

  • A custom Kayak tool to translate the user input into a valid Kayak search URL
  • A Browserbase tool to navigate on Kayak and interact with the web page

Finally, we will define 2 tasks: “Search Flights” and “Search Booking Providers”.

We can visualize our Flight Booking Crew as follows:

Our Crew comprises 2 Agents, 2 Tools, and 2 Tasks.

Let’s implement our Crew!

​
2. Installation

Let’s setup the project by installing the required dependencies:

pip install crewai 'crewai[tools]' html2text playwright python-dotenv

Create a .env file with the following variables and their respective values:

.env
OPENAI_API_KEY=
BROWSERBASE_API_KEY=
BROWSERBASE_PROJECT_ID=
# our Flight Booking's "Search Flights" Agent will have to load a lot of context (heavy webpages as text),
# let's configure a specific OpenAI model to avoid token size limits:
OPENAI_MODEL_NAME=gpt-4-turbo

Where can I find my OpenAI and Browserbase API Keys?

​
3. Create the Tools

While CrewAI provides a wide range of tools (e.g., the SerperDevTool to perform searches with Google Search), our “Search Flights” Agent needs 2 custom tools:

  • a custom Kayak tool to assemble a valid Kayak search URL
  • a Browserbase loader to navigate and interact with the web pages

​
The Browserbase Tool

The Kayak website relies heavily on JavaScript and performs a live flight search, making it hard to interact with:

The page is fully loaded, however the flights are still being searched.

Fortunately, leveraging Browserbase’s headless browsers makes loading and interacting with such websites easier while benefiting from its Stealth features.

Let’s take a look at our custom Browserbase Tool implementation:

browserbase.py
import os
from crewai_tools import tool
from playwright.sync_api import sync_playwright
from html2text import html2text
from time import sleep


@tool("Browserbase tool")
def browserbase(url: str):
    """
    Loads a URL using a headless webbrowser

    :param url: The URL to load
    :return: The text content of the page
    """
    with sync_playwright() as playwright:
        browser = playwright.chromium.connect_over_cdp(
            "wss://connect.browserbase.com?apiKey="
            + os.environ["BROWSERBASE_API_KEY"]
        )
        context = browser.contexts[0]
        page = context.pages[0]
        page.goto(url)

        # Wait for the flight search to finish
        sleep(25)

        content = html2text(page.content())
        browser.close()
        return content

Custom Tool definition

A custom Tool is composed of 3 elements:

  • a name, via the @tool("name") decorator
  • a description defining the purpose of the tool along with its parameters
  • a function that contains the tool’s logic

The description, provided as a multi-line comment, is used by the Agents to evaluate the best-fitted Tool to help complete a given Task.

A description can also provide instructions on the parameters. Here, we instruct that the unique url parameter should be a URL.

Browserbase Tool Logic

The Browserbase tool utilizes the playwright library along with the Browserbase Connect API to initiate a headless browser session. This setup allows interaction with web pages as follows:

browser = playwright.chromium.connect_over_cdp(
    "wss://connect.browserbase.com?apiKey="
    + os.environ["BROWSERBASE_API_KEY"]
)

Then, it leverages the html2text library to convert the webpage’s content to text and return it to the Agent for processing.

​
The Kayak Tool

Agents are capable of reasoning but cannot build a valid Kayak search URL from the ground up.

To help our “Flights” Agent, we will create a simple Kayak Tool below:

kayak.py
from crewai_tools import tool
from typing import Optional

@tool("Kayak tool")
def kayak(
    departure: str, destination: str, date: str, return_date: Optional[str] = None
) -> str:
    """
    Generates a Kayak URL for flights between departure and destination on the specified date.

    :param departure: The IATA code for the departure airport (e.g., 'SOF' for Sofia)
    :param destination: The IATA code for the destination airport (e.g., 'BER' for Berlin)
    :param date: The date of the flight in the format 'YYYY-MM-DD'
    :return_date: Only for two-way tickets. The date of return flight in the format 'YYYY-MM-DD'
    :return: The Kayak URL for the flight search
    """
    print(f"Generating Kayak URL for {departure} to {destination} on {date}")
    URL = f"https://www.kayak.com/flights/{departure}-{destination}/{date}"
    if return_date:
        URL += f"/{return_date}"
    URL += "?currency=USD"
    return URL

The Kayak tool describes multiple parameters with specific format instructions.

For example: date: The date of the flight in the format 'YYYY-MM-DD'

This illustrates the flexibility of Tools that can rely on the Agents powerful reasoning capabilities to solve formatting challenges that generally require some preprocessing.

​
4. Set up the Agents

Our Flights Agent now has the tools to navigate the Kayak website from a high-level user input (“San Francisco to New York one-way on 21st September”).

Let’s now set up our 2 Agents:

main.py
from crewai import Agent
# import our tools
from browserbase import browserbase
from kayak import kayak


flights_agent = Agent(
    role="Flights",
    goal="Search flights",
    backstory="I am an agent that can search for flights.",
    tools=[kayak, browserbase],
    allow_delegation=False,
)

summarize_agent = Agent(
    role="Summarize",
    goal="Summarize content",
    backstory="I am an agent that can summarize text.",
    allow_delegation=False,
)

As outlined in the introduction, an Agent needs 3 properties: a role, a goal, and a backstory.

The role of our two Agents is to orchestrate the tools (build the URL, then navigate to it) and extract the information from the webpages’ text. For this reason, their definition is straightforward.

What is the role of the Summarize Agent?

Through our iterations in building this Flight Booker, we realized that the Crew, with a single Flights Agent was struggling to distinguish flights from flight providers (booking links).

The Summarize Agent, as we will cover in the next section, is not assigned to any task. It is created and assigned to the Crew to help digest the text extracted from the web pages and distinguish the flights from the providers (booking links).

​
4. Define the Tasks

Let’s now define the core part of our Flight Booking Crew, the Tasks.

From a given flight criteria, our Crew should print the 5 first available flights with their associated booking link. To achieve such a result, our Crew needs to:

  1. Navigate to the Kayak search URL and extract the top 5 flights
  2. For each flight, navigate to the flight details URL to extract the available providers and booking links

​
The “Search flights” Task

Our Search flights Task is bound to our Flights Agent, getting access to our custom tools:

main.py
from crewai import Task

# Agents definitions...

output_search_example = """
Here are our top 5 flights from San Francisco to New York on 21st September 2024:
1. Delta Airlines: Departure: 21:35, Arrival: 03:50, Duration: 6 hours 15 minutes, Price: $125, Details: https://www.kayak.com/flights/sfo/jfk/2024-09-21/12:45/13:55/2:10/delta/airlines/economy/1
"""

search_task = Task(
    description=(
        "Search flights according to criteria {request}. Current year: {current_year}"
    ),
    expected_output=output_search_example,
    agent=flights_agent,
)

The description will be provided to the Flights Agent who will call:

  1. The Kayak Tool to build a valid Kayak search URL
  2. Then, leverage the Browserbase Tool to get the flight results as text
  3. Finally, using the output_search_example and with the help of the Summarize Agent, it will return a list of 5 flights

Why do we provide the current_year?

Most users will prompt a relative date, for example: “San Francisco to New York one-way on 21st September”.

An Agent’s reasoning relies on OpenAI that lacks some intuition on relative date (OpenAI will always think we are in 2022).

For this reason, we need to specify the current year in the prompt (Task’s description).

​
The “Search Booking Providers” Task

The Search Booking Providers Task relies heavily on the Agent reasoning capabilities:

main.py
from crewai import Task

# Agents definitions...

output_providers_example = """
Here are our top 5 picks from San Francisco to New York on 21st September 2024:
1. Delta Airlines:
    - Departure: 21:35
    - Arrival: 03:50
    - Duration: 6 hours 15 minutes
    - Price: $125
    - Booking: [Delta Airlines](https://www.kayak.com/flights/sfo/jfk/2024-09-21/12:45/13:55/2:10/delta/airlines/economy/1)
    ...
"""

search_booking_providers_task = Task(
    description="Load every flight individually and find available booking providers",
    expected_output=output_providers_example,
    agent=flights_agent,
)

By asking to “Load every flight individually”, the Flights Agent will understand that it needs to locate a URL to navigate to for each flight result.

The Search Booking Providers will indirectly rely on the Summarize Agent to consolidate the flights result and individual flight providers’ results as showcased in output_providers_example.

​
4. Assemble our Flight Booking Crew

It is time to assemble our Crew by arranging the Task in the correct order (search flights, then gather providers and booking links):

main.py
import sys
import datetime
from crewai import Crew, Process, Task, Agent
from browserbase import browserbase
from kayak import kayak
from dotenv import load_dotenv

load_dotenv()  # take environment variables from .env.

# Tasks and Agents definitions...

crew = Crew(
    agents=[flights_agent, summarize_agent],
    tasks=[search_task, search_booking_providers_task],
    # let's cap the number of OpenAI requests as the Agents
    #   may have to do multiple costly calls with large context
    max_rpm=100,
    # let's also set verbose=True and planning=True
    #   to see the progress of the Agents
    #   and the Task execution. Remove these lines
    #   if you want to run the script without
    #   seeing the progress (like in production).
    verbose=True,
    planning=True,
)

result = crew.kickoff(
    inputs={
        "request": sys.argv[1],
        "current_year": datetime.date.today().year,
    }
)

print(result)

The Crew must complete the Search Flight task followed by the Search Booking Providers task.

As covered earlier, the Summarize Agent gets assigned to the Crew - not to a Task - to help consolidate the flights and providers into a simple list.

Let the Crew kick off!

A Crew process starts by calling the kickoff() method.

Our Crew needs 2 inputs: the user input (“San Francisco to New York one-way on 21st September”) and the current year.

Our CrewAI program is now complete!

Let’s give it a try and look at its execution steps in detail.

​
Running the program

OpenAI cost

Expect each run of the program to cost around $0.50 OpenAI credits.

The Agent reasoning relies heavily on OpenAI and sends large chunks of text (the webpages), resulting in significant contexts (~50k context tokens per run).

Let’s search for a one-way flight from New York to San Francisco by running:

python3 main.py "San Francisco to New York one-way on 21st September"

As the program starts running in verbose mode, you should see some logs stream in your terminal; let’s take a closer look at the steps.

​
A close look at the Crew steps

Looking at the debugging logs streamed to the terminal helps us understand how our crew works.

Let’s explore the logs in the following steps:

Once finished, our program prints the final answered returned by the Crew:

Here are our top 5 flights from San Francisco (SFO) to Newark (EWR) on September 21, 2024:

1. **Alaska Airlines**:
   - Departure: 8:50 am
   - Arrival: 5:24 pm
   - Duration: 5 hours 34 minutes
   - Layovers: Nonstop
   - Price: $125
   - Booking: [Alaska Airlines Saver](https://www.kayak.com/book/flight?code=noAiOYx8xU.4fFBlTtfVpoDzQq2dWkU9A.12411.28f6c8a3257adb48c2f7d8207660b2a0&h=41a638bca25d&_kw_pbranded=true&sub=F-1450586051791345216E0040d85ce85&pageOrigin=F..RP.FE.M4)

2. **United Airlines**:
   - Departure: 1:30 pm
   - Arrival: 9:50 pm
   - Duration: 5 hours 20 minutes
   - Layovers: Nonstop
   - Price: $125
   - Booking: [United Airlines Basic Economy](https://www.kayak.com/book/flight?code=noAiOYx8xU.UYIuDTZHiSY.12448.df899b8e44c813d2f8c5501a1648fc15&h=3e1b76440249&sub=F-5023348394153941183E0bc6c2fafa5&pageOrigin=F..RP.FE.M1)

3. **United Airlines**:
   - Departure: 4:40 pm
   - Arrival: 1:13 am+1
   - Duration: 5 hours 33 minutes
   - Layovers: Nonstop
   - Price: $125
   - Booking: [United Airlines Basic Economy](https://www.kayak.com/book/flight?code=noAiOYx8xU.UYIuDTZHiSY.12448.5ec6fd14128fd0c540fd0f53d711947a&h=f6ae82999387&sub=F-5023348393135040028E0bc6c2fafa5&pageOrigin=F..RP.FE.M6)

4. **United Airlines**:
   - Departure: 11:59 pm
   - Arrival: 8:27 am+1
   - Duration: 5 hours 28 minutes
   - Layovers: Nonstop
   - Price: $144
   - Booking: [United Airlines Basic Economy](https://www.kayak.com/book/flight?code=noAiOYx8xU.UYIuDTZHiSY.14383.65a16596bc682cce98ddcd39666710a3&h=e34e775c0ed7&sub=F-5023348391216069073E0bc6c2fafa5&pageOrigin=F..RP.FE.M9)

5. **United Airlines**:
   - Departure: 7:15 am
   - Arrival: 3:30 pm
   - Duration: 5 hours 15 minutes
   - Layovers: Nonstop
   - Price: $159
   - Booking: [United Airlines Basic Economy](https://www.kayak.com/book/flight?code=noAiOYx8xU.UYIuDTZHiSY.15888.f2fb6ff5bafca7eed4751036a9b91597&h=7ce06a5da162&sub=F-5023348394219198114E0bc6c2fafa5&pageOrigin=F..RP.FE.M10)

​
Wrapping up

CrewAI provides a powerful way to develop AI Agents. The traditional approach of Prompt Engineering is replaced by instructions that leverage the Agent’s reasoning capabilities.

As we covered in this example, the Agents are capable of completing Tasks defined with high-level instructions (ex: “Load every flight individually and find available booking providers”)

Combined with Browserbase headless browsers, crewAI helps create powerful AI Agents that automate human tasks or provide support in accessing data not accessible through public APIs.

View the source code on GitHub

Check out the repo!