Master Multi-Agent Systems with CrewAI
Create powerful, collaborative AI systems that can tackle complex tasks through agent cooperation and specialization
1. Introduction to CrewAI
What is CrewAI?
CrewAI is a cutting-edge Python framework designed for orchestrating role-playing, autonomous AI agents. It enables the creation of specialized AI teams where agents can take on different roles, make autonomous decisions, and collaborate to solve complex problems. Unlike single-agent systems, CrewAI focuses on collaborative intelligence, allowing for more sophisticated problem-solving through agent specialization and teamwork.
Why Use CrewAI?
Role-Playing Agents
Agents can assume specific personas with unique goals, backstories, and expertise areas, enabling more specialized task performance.
Autonomous Decision Making
Agents can make decisions independently based on their goals and available information, reducing the need for human intervention.
Seamless Collaboration
Agents can work together, sharing information and building upon each other's outputs to tackle complex, multi-stage tasks.
Complex Problem Solving
The framework is designed to handle intricate workflows, decision trees, and multi-stage problems that would be challenging for single agents.
Key Features of CrewAI
- Independent Framework: Built from scratch without dependencies on other agent frameworks like LangChain.
- Flexible Agent Design: Create agents with customized roles, goals, tools, and personalities.
- Process Flexibility: Choose between sequential and hierarchical process flows for task execution.
- Tool Integration: Integrate a wide range of tools to enhance agent capabilities for specific tasks.
- Memory Systems: Incorporate short-term and long-term memory to enhance agent performance over time.
- Output Formatting: Structured outputs in various formats including raw text, JSON, and Pydantic models.
2. Installation & Setup
Installing CrewAI
Install CrewAI using pip:
pip install crewai
For additional tools support:
pip install crewai-tools
Environment Setup
CrewAI works with various language models. For OpenAI models, set up your API key:
# In your .env file
OPENAI_API_KEY=your-api-key-here
For local models using Ollama:
# Example setup for Ollama
from langchain.llms import Ollama
local_llm = Ollama(model="llama2")
Project Creation
Use CrewAI's CLI to create a new project:
pip install crewai-cli
crewai create my_crew_project
This creates a project structure with the following elements:
my_crew_project/
├── src/
│ └── my_crew_project/
│ ├── config/
│ │ ├── agents.yaml
│ │ └── tasks.yaml
│ ├── crew.py
│ └── main.py
├── pyproject.toml
└── README.md
Tip: Use virtual environments to manage your dependencies and avoid conflicts with other projects. Consider using tools like venv
, virtualenv
, or conda
.
3. Core Concepts
CrewAI is built around four main concepts: Agents, Tasks, Crews, and Tools. Understanding these components and how they interact is essential for building effective multi-agent systems.
3.1 Agents
An agent in CrewAI is an autonomous unit programmed to perform tasks, make decisions, and communicate with other agents. Each agent has a specific role, goal, and backstory that shape its behavior and decision-making process.
Key Agent Attributes
Attribute |
Description |
Required |
role |
Defines the agent's function within the crew |
Yes |
goal |
The individual objective that the agent aims to achieve |
Yes |
backstory |
Provides context to the agent's role and goal |
Yes |
llm |
The language model that powers the agent |
No |
tools |
Functions/capabilities the agent can use |
No |
verbose |
Controls the level of logging detail |
No |
allow_delegation |
Enables the agent to delegate tasks to others |
No |
Creating an Agent
from crewai import Agent
researcher = Agent(
role="Research Analyst",
goal="Find the most accurate and up-to-date information on AI trends",
backstory="You're a senior research analyst with 15 years of experience in technology trends analysis. Your attention to detail and ability to distinguish relevant information from noise is unmatched.",
tools=[search_tool], # Optional
llm=openai_llm, # Optional
verbose=True # Optional
)
YAML Configuration (Recommended Approach)
Define agents in YAML for better organization:
# In agents.yaml
researcher:
role: "Research Analyst"
goal: "Find the most accurate and up-to-date information on AI trends"
backstory: "You're a senior research analyst with 15 years of experience in technology trends analysis. Your attention to detail and ability to distinguish relevant information from noise is unmatched."
llm: "openai" # Reference to a defined LLM
writer:
role: "Technical Writer"
goal: "Transform complex research into clear, engaging content"
backstory: "You're an experienced technical writer who can explain complex concepts in simple terms."
llm: "anthropic" # Reference to a defined LLM
Then in your code:
@agent
def researcher(self) -> Agent:
return Agent(
config=self.agents_config["researcher"],
tools=[search_tool]
)
Best Practices for Agents:
- Give agents clear, specific goals that guide their actions
- Create detailed backstories that provide context for decision-making
- Define specialized roles with minimal overlap to improve collaboration
- Equip agents with appropriate tools for their specific roles
3.2 Tasks
Tasks are specific assignments completed by agents. They define what needs to be done, how it should be accomplished, and what the expected output is. Tasks can be assigned to specific agents and can depend on the outputs of other tasks.
Key Task Attributes
Attribute |
Description |
Required |
description |
Detailed instructions for the task |
Yes |
expected_output |
Description of what the completed task should produce |
Yes |
agent |
The agent responsible for executing the task |
No* |
tools |
Specific tools available for this task |
No |
context |
Outputs from previous tasks to use as context |
No |
async_execution |
Whether to execute the task asynchronously |
No |
human_input |
Whether human review is required for final answer |
No |
output_file |
File path for storing the task output |
No |
output_json |
Pydantic model for JSON output structure |
No |
output_pydantic |
Pydantic model for output validation |
No |
* Either agent must be specified when creating the task, or assigned when adding the task to a crew.
Creating a Task
from crewai import Task
research_task = Task(
description="Research the latest advancements in quantum computing and their potential applications in cryptography. Focus on breakthroughs from the past 12 months.",
expected_output="A comprehensive 2-page report detailing recent quantum computing advances and their implications for cryptographic systems.",
agent=researcher,
tools=[search_tool, browse_tool], # Optional
context=[], # Optional
async_execution=False # Optional
)
YAML Configuration
# In tasks.yaml
research_task:
description: "Research the latest advancements in quantum computing and their potential applications in cryptography. Focus on breakthroughs from the past 12 months."
expected_output: "A comprehensive 2-page report detailing recent quantum computing advances and their implications for cryptographic systems."
agent: researcher # Reference to agent defined in agents.yaml
write_article_task:
description: "Create an engaging blog article about quantum computing advances aimed at a technical audience with some knowledge of cryptography."
expected_output: "A 1500-word blog article with sections, subheadings, and easily digestible explanations of complex concepts."
agent: writer
context:
- research_task # Reference to another task whose output will be used
Task Dependencies & Context
One of CrewAI's powerful features is the ability to create task dependencies:
from crewai import Task
research_task = Task(
description="Research quantum computing advances",
expected_output="Research report",
agent=researcher
)
write_article_task = Task(
description="Write an engaging article based on the research findings",
expected_output="1500-word article",
agent=writer,
context=[research_task] # This task depends on research_task
)
Structured Output
For more controlled outputs, use Pydantic models:
from pydantic import BaseModel
from crewai import Task
class ResearchReport(BaseModel):
title: str
summary: str
findings: list[str]
implications: list[str]
research_task = Task(
description="Research quantum computing advances",
expected_output="Structured research report",
agent=researcher,
output_pydantic=ResearchReport # Output will conform to this model
)
Best Practices for Tasks:
- Write clear, specific task descriptions with unambiguous instructions
- Define concrete expected outputs that can be measured
- Use task context to create logical workflows between agents
- Consider using structured outputs for complex data
- Break complex processes into multiple distinct tasks
3.3 Crews
A Crew in CrewAI orchestrates the collaboration between agents to complete a set of tasks. It defines the strategy for task execution, manages the communication between agents, and handles the workflow from start to finish.
Key Crew Attributes
Attribute |
Description |
Required |
agents |
List of agents participating in the crew |
Yes |
tasks |
List of tasks to be completed by the crew |
Yes |
process |
Workflow strategy (sequential/hierarchical) |
No |
verbose |
Controls level of logging and output |
No |
manager_llm |
LLM used by the manager in hierarchical process |
No* |
function_calling_llm |
LLM used for function/tool calling |
No |
memory |
Enables storing execution memories |
No |
cache |
Enables caching tool execution results |
No |
full_output |
Return outputs from all tasks, not just the last |
No |
* Required when using hierarchical process
Creating a Crew
from crewai import Crew
crew = Crew(
agents=[researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
process="sequential", # Optional, default is sequential
verbose=True, # Optional
memory=True, # Optional
cache=True # Optional
)
Process Types
Sequential Process
Tasks are executed one after another in the order they are defined. This is the default process type.
Simple to understand
Predictable flow
Hierarchical Process
A manager agent coordinates the crew, delegating tasks and validating outcomes. Requires a manager_llm.
More flexible
Complex interactions
Kicking Off a Crew
result = crew.kickoff()
print(result) # Prints the final output
# For hierarchical process
hierarchical_crew = Crew(
agents=[manager, researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
process="hierarchical",
manager_llm=openai_llm # Required for hierarchical process
)
result = hierarchical_crew.kickoff()
Working with Crew Outputs
result = crew.kickoff()
# Access the raw text output
print(result.raw)
# If using output_json or output_pydantic in the final task
if result.json_dict:
print(f"Title: {result.json_dict['title']}")
print(f"Summary: {result.json_dict['summary']}")
# Access all task outputs (if full_output=True)
for task_output in result.tasks_output:
print(f"Task: {task_output.description}")
print(f"Output: {task_output.raw}")
# Check token usage
print(f"Token usage: {result.token_usage}")
Best Practices for Crews:
- Start with sequential processes for simpler workflows
- Use hierarchical processes when tasks require complex coordination
- Enable memory for tasks that benefit from context retention
- Use verbose mode during development for debugging
- Consider the crew as a reusable workflow that can be applied to different inputs
3.4 Tools
Tools extend the capabilities of agents, allowing them to perform actions like searching the web, analyzing data, or interacting with external systems. CrewAI supports a wide range of tools from both its own toolkit and LangChain.
Popular CrewAI Tools
Tool Category |
Examples |
Use Cases |
Search Tools |
SerperDevTool, WebsiteSearchTool |
Information gathering, research |
Document Tools |
PDFSearchTool, CSVSearchTool, DOCXSearchTool |
Extracting information from documents |
Web Tools |
ScrapeWebsiteTool, FirecrawlSearchTool |
Extracting data from websites |
Code & Data Tools |
CodeInterpreterTool, CodeDocsSearchTool |
Processing code, analyzing data |
Media Tools |
DALL-E Tool, Vision Tool, YoutubeVideoSearchTool |
Generating and analyzing media |
RAG Tools |
RagTool, Various document search tools |
Retrieval augmented generation |
Installing Tools
pip install crewai-tools
Using Tools
from crewai_tools import SerperDevTool, ScrapeWebsiteTool
# Initialize tools
search_tool = SerperDevTool()
scrape_tool = ScrapeWebsiteTool()
# Assign tools to an agent
researcher = Agent(
role="Research Analyst",
goal="Find accurate information",
backstory="You're an expert researcher",
tools=[search_tool, scrape_tool]
)
Creating Custom Tools
There are two main ways to create custom tools in CrewAI:
1. Using the tool decorator
from crewai.tools import tool
@tool
def calculate_mortgage(principal: float, interest_rate: float, years: int) -> str:
"""
Calculate the monthly mortgage payment.
Args:
principal: Loan amount in dollars
interest_rate: Annual interest rate (as a decimal, e.g., 0.05 for 5%)
years: Loan term in years
Returns:
Monthly payment amount and total interest paid
"""
# Tool implementation here
monthly_rate = interest_rate / 12
num_payments = years * 12
# Calculate monthly payment using the mortgage formula
payment = principal * (monthly_rate * (1 + monthly_rate)**num_payments) / ((1 + monthly_rate)**num_payments - 1)
total_paid = payment * num_payments
total_interest = total_paid - principal
return f"Monthly payment: ${payment:.2f}\nTotal interest paid: ${total_interest:.2f}"
2. Subclassing BaseTool
from crewai.tools import BaseTool
from pydantic import Field
from typing import Dict, Any
class StockPriceTool(BaseTool):
"""Tool for getting stock price information."""
name: str = "Stock Price Tool"
description: str = "Get current stock price information for a given ticker symbol"
def _run(self, ticker: str) -> Dict[str, Any]:
"""
Get stock price information.
Args:
ticker: The stock ticker symbol (e.g., AAPL for Apple)
Returns:
Dictionary with stock information
"""
# Implement API call or data retrieval here
# This is a simplified example
import random
price = round(random.uniform(50, 500), 2)
return {
"ticker": ticker,
"price": price,
"currency": "USD",
"timestamp": "2023-10-24T15:30:00Z"
}
Tool Caching
CrewAI tools support caching to improve performance:
from crewai import Agent
# Enable caching at the agent level
researcher = Agent(
role="Research Analyst",
goal="Find accurate information",
backstory="You're an expert researcher",
tools=[search_tool, scrape_tool],
cache=True # Enable caching for all tools used by this agent
)
Important: When using tools that require API keys (like SerperDevTool), make sure to set up the appropriate environment variables before using them.
4. Building Your First Crew
Let's walk through building a basic research crew that will research a topic and create a report. This example demonstrates the key concepts of CrewAI in a practical application.
Step 1: Project Setup
1
Create a new project directory and set up your environment:
mkdir research-crew
cd research-crew
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install crewai crewai-tools
Step 2: Define Your Agents
2
Create a file named agents.py
to define your agents:
from crewai import Agent
def create_researcher(llm):
return Agent(
role="Research Analyst",
goal="Find comprehensive and accurate information on the given topic",
backstory="You are an experienced research analyst with a talent for finding reliable information quickly. You're meticulous about fact-checking and always seek multiple sources to confirm information.",
llm=llm,
verbose=True
)
def create_writer(llm):
return Agent(
role="Content Writer",
goal="Transform research findings into clear, engaging, and well-structured reports",
backstory="You are a talented writer with a knack for explaining complex topics in an accessible way. You have experience creating various types of content including reports, articles, and presentations.",
llm=llm,
verbose=True
)
Step 3: Define Your Tasks
3
Create a file named tasks.py
to define your tasks:
from crewai import Task
from crewai_tools import SerperDevTool
search_tool = SerperDevTool()
def create_research_task(researcher, topic):
return Task(
description=f"Research the topic: {topic}. Find key information including recent developments, major concepts, important figures, and relevant statistics. Focus on high-quality sources and make sure to fact-check the information.",
expected_output="A comprehensive research document with organized sections covering different aspects of the topic. Include a list of sources.",
agent=researcher,
tools=[search_tool]
)
def create_report_task(writer, topic):
return Task(
description=f"Create a well-structured report on {topic} based on the research findings. The report should be informative, engaging, and easy to understand.",
expected_output="A polished report with an executive summary, introduction, main body (with sections and subsections), conclusion, and references.",
agent=writer
)
Step 4: Set Up Your Crew
4
Create a file named crew.py
to set up your crew:
from crewai import Crew
from agents import create_researcher, create_writer
from tasks import create_research_task, create_report_task
def create_research_crew(llm, topic):
# Create agents
researcher = create_researcher(llm)
writer = create_writer(llm)
# Create tasks
research_task = create_research_task(researcher, topic)
report_task = create_report_task(writer, topic)
report_task.context = [research_task] # Link tasks
# Create and return the crew
crew = Crew(
agents=[researcher, writer],
tasks=[research_task, report_task],
verbose=True
)
return crew
Step 5: Create the Main Script
5
Create a file named main.py
to run your crew:
import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from crew import create_research_crew
def main():
# Load environment variables
load_dotenv()
# Set up the language model
llm = ChatOpenAI(
api_key=os.getenv("OPENAI_API_KEY"),
model="gpt-4",
temperature=0.5
)
# Topic to research
topic = "Quantum Computing Basics"
# Create and run the crew
crew = create_research_crew(llm, topic)
result = crew.kickoff()
# Save the output to a file
with open("report.md", "w") as f:
f.write(result.raw)
print(f"Research report saved to report.md")
if __name__ == "__main__":
main()
Step 6: Set Up Environment Variables
6
Create a .env
file with your API keys:
OPENAI_API_KEY=your_openai_api_key_here
SERPER_API_KEY=your_serper_api_key_here
Step 7: Run Your Crew
7
Run the main script to execute your crew:
python main.py
You'll see the agents working, thinking, and communicating in the terminal output. When complete, check the generated report.md file for the final output.
Success!
You've created your first CrewAI crew with:
- Two specialized agents with distinct roles
- A sequential workflow where tasks build on each other
- Integration of external tools for research
- A complete end-to-end process from research to report generation
5. Advanced Features
CrewAI offers several advanced features that can enhance your multi-agent systems. Let's explore some of these powerful capabilities.
Hierarchical Process
The hierarchical process introduces a manager agent that coordinates the crew's activities, making decisions about task delegation and validation.
from crewai import Crew
from langchain_openai import ChatOpenAI
manager_llm = ChatOpenAI(model="gpt-4")
crew = Crew(
agents=[researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
process="hierarchical",
manager_llm=manager_llm, # Required for hierarchical process
verbose=True
)
result = crew.kickoff()
Custom Manager Agent
You can define a custom manager agent with specialized expertise for better coordination:
from crewai import Agent, Crew
project_manager = Agent(
role="Project Manager",
goal="Efficiently coordinate the team to deliver high-quality outputs on time",
backstory="You're an experienced project manager with a track record of successful project deliveries. You excel at resource allocation, risk management, and ensuring team synergy.",
llm=openai_llm,
verbose=True
)
crew = Crew(
agents=[researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
process="hierarchical",
manager_agent=project_manager, # Using custom manager
verbose=True
)
Memory Systems
CrewAI supports memory to enable agents to retain information across interactions:
from crewai import Crew
crew = Crew(
agents=[researcher, writer],
tasks=[research_task, writing_task],
memory=True, # Enable memory
verbose=True
)
# Memory is particularly useful for multi-session interactions
result_day1 = crew.kickoff(inputs={"topic": "Quantum Computing Part 1"})
# Later session continues with awareness of previous work
result_day2 = crew.kickoff(inputs={"topic": "Quantum Computing Part 2"})
Structured Outputs
Get structured data from your crews using Pydantic models:
from pydantic import BaseModel, Field
from typing import List
from crewai import Task
class ResearchFindings(BaseModel):
topic: str = Field(description="The main topic of research")
key_points: List[str] = Field(description="List of main findings or points")
sources: List[str] = Field(description="References and sources of information")
future_directions: List[str] = Field(description="Suggested areas for further research")
research_task = Task(
description="Research quantum computing advances",
expected_output="Structured research findings",
agent=researcher,
output_pydantic=ResearchFindings
)
result = crew.kickoff()
print(f"Topic: {result.pydantic.topic}")
print(f"Key points: {result.pydantic.key_points}")
print(f"Sources: {result.pydantic.sources}")
Asynchronous Execution
Run tasks asynchronously for better performance:
from crewai import Task, Crew
# Create tasks with async_execution flag
async_research_task = Task(
description="Research quantum computing",
expected_output="Research findings",
agent=researcher,
async_execution=True # This task will run asynchronously
)
# Use the async kickoff method
import asyncio
async def run_crew_async():
crew = Crew(
agents=[researcher, writer],
tasks=[async_research_task, writing_task]
)
result = await crew.kickoff_async()
return result
# Run the async function
result = asyncio.run(run_crew_async())
Task Guardrails
Implement validation for task outputs:
from crewai import Task
from typing import Tuple, Any
def validate_research(output: str) -> Tuple[bool, Any]:
"""Validate research output meets quality criteria."""
# Check for minimum length
if len(output) < 500:
return False, "Research output is too short. Please provide more detailed information."
# Check for sources
if "References:" not in output and "Sources:" not in output:
return False, "Missing references or sources. Please include sources for your research."
# If all checks pass
return True, output
research_task = Task(
description="Research quantum computing advances",
expected_output="Comprehensive research with sources",
agent=researcher,
guardrail=validate_research # Add validation function
)
Task Callbacks
Execute custom functions after tasks complete:
from crewai import Task, Crew
def post_research_callback(task_output):
"""Process research after completion."""
print(f"Research task completed with {len(task_output.raw)} characters")
# Save to database, send notification, etc.
with open("research_results.md", "w") as f:
f.write(task_output.raw)
return task_output
research_task = Task(
description="Research quantum computing advances",
expected_output="Comprehensive research with sources",
agent=researcher,
callback=post_research_callback # Will execute after task completion
)
# You can also set a callback for the entire crew
def crew_step_callback(step_output):
"""Track each step of crew execution."""
print(f"Step completed: {step_output}")
return step_output
crew = Crew(
agents=[researcher, writer],
tasks=[research_task, writing_task],
step_callback=crew_step_callback
)
Planning
Enable automatic planning for more complex workflows:
from crewai import Crew
crew = Crew(
agents=[researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
planning=True, # Enable automatic planning
planning_llm=openai_llm, # Optional, specify LLM for planning
verbose=True
)
result = crew.kickoff()
Tip: Advanced features often require more careful configuration and testing. Start with simpler setups and gradually incorporate advanced features as you become more familiar with CrewAI.
6. Example Applications
Let's explore some practical example applications that you can build with CrewAI. These examples demonstrate how to apply CrewAI's capabilities to solve real-world problems.
Business Plan Generator
Use Case: Generate a business plan for a startup idea
Agents:
- Market Research Analyst: Researches market trends, competitors, and customer needs
- Technologist: Evaluates technical feasibility and implementation requirements
- Business Consultant: Creates the final business plan with financial projections
Workflow:
- Market Research Analyst analyzes the market demand for the proposed product
- Technologist evaluates the technology requirements based on market analysis
- Business Consultant creates a comprehensive business plan based on both analyses
Example input: "Generate a business plan for an AI-powered personal fitness coaching app that uses computer vision to track workouts and provide real-time feedback."
Content Creation System
Use Case: Create blog posts, social media content, and email newsletters
Agents:
- Topic Researcher: Identifies trending topics and key information
- Content Writer: Creates the main content in an engaging style
- Editor: Reviews content for clarity, accuracy, and SEO optimization
- Social Media Specialist: Crafts promotional snippets for different platforms
Workflow:
- Topic Researcher identifies trending topics and gathers key information
- Content Writer creates the main content based on research
- Editor reviews and refines the content
- Social Media Specialist creates promotional material for different platforms
Example input: "Create content about sustainable living tips for urban apartment dwellers, including a blog post, Twitter thread, and newsletter."
Trip Planner
Use Case: Create personalized travel itineraries
Agents:
- Destination Researcher: Researches attractions, accommodations, and local information
- Travel Logistics Expert: Plans transportation, timing, and practical arrangements
- Experience Curator: Personalizes the itinerary based on traveler preferences
Workflow:
- Destination Researcher gathers information about the location
- Travel Logistics Expert creates a feasible schedule with transportation
- Experience Curator customizes the plan based on preferences and provides the final itinerary
Example input: "Plan a 5-day trip to Tokyo for a family of four with children aged 8 and 12 who are interested in technology, anime, and outdoor activities. Budget: moderate."
Market Analysis Report
Use Case: Analyze a specific market or industry
Agents:
- Industry Researcher: Gathers data on market trends, size, and growth
- Competitive Analyst: Analyzes key competitors and market dynamics
- Financial Analyst: Reviews financial aspects and investment potential
- Report Writer: Creates the final comprehensive report
Workflow:
- Industry Researcher collects market data and trends
- Competitive Analyst evaluates the competitive landscape
- Financial Analyst reviews the financial potential and risks
- Report Writer compiles all analyses into a cohesive report
Example input: "Create a market analysis report for the electric vehicle charging infrastructure industry in Europe, focusing on growth opportunities for the next 5 years."
Tip for Implementation: Start with a simplified version of these examples with just 2-3 agents, then expand as you become more comfortable with CrewAI. Focus on creating well-defined roles and clear task descriptions to ensure effective agent collaboration.
7. CrewAI Cheatsheet
This comprehensive cheatsheet provides quick reference for the most common CrewAI patterns and configurations.
Agent Creation Patterns
# Basic Agent
from crewai import Agent
basic_agent = Agent(
role="Researcher",
goal="Find accurate information",
backstory="You're an experienced researcher with expertise in data analysis."
)
# Agent with Tools
from crewai_tools import SerperDevTool
search_tool = SerperDevTool()
agent_with_tools = Agent(
role="Researcher",
goal="Find accurate information",
backstory="You're an experienced researcher.",
tools=[search_tool]
)
# Agent with Custom LLM
from langchain_openai import ChatOpenAI
custom_llm = ChatOpenAI(model="gpt-4")
agent_with_llm = Agent(
role="Researcher",
goal="Find accurate information",
backstory="You're an experienced researcher.",
llm=custom_llm
)
# Agent with Delegation
delegating_agent = Agent(
role="Team Manager",
goal="Coordinate research activities",
backstory="You're a team manager with excellent delegation skills.",
allow_delegation=True
)
Task Creation Patterns
# Basic Task
from crewai import Task
basic_task = Task(
description="Research quantum computing advances",
expected_output="Comprehensive research report",
agent=researcher
)
# Task with Tools
task_with_tools = Task(
description="Research quantum computing advances",
expected_output="Comprehensive research report",
agent=researcher,
tools=[search_tool, web_scraper_tool]
)
# Task with Context (Dependency)
dependent_task = Task(
description="Write an article based on research findings",
expected_output="1500-word article",
agent=writer,
context=[research_task] # This task depends on research_task
)
# Task with Structured Output
from pydantic import BaseModel
from typing import List
class ResearchReport(BaseModel):
title: str
key_findings: List[str]
sources: List[str]
structured_task = Task(
description="Research quantum computing advances",
expected_output="Structured research report",
agent=researcher,
output_pydantic=ResearchReport
)
# Task with File Output
file_output_task = Task(
description="Generate a research report",
expected_output="Comprehensive markdown report",
agent=researcher,
output_file="output/research_report.md"
)
# Asynchronous Task
async_task = Task(
description="Research quantum computing advances",
expected_output="Research report",
agent=researcher,
async_execution=True
)
# Task with Human Input
human_review_task = Task(
description="Create final research report",
expected_output="Polished report",
agent=editor,
human_input=True
)
Crew Creation Patterns
# Sequential Crew (Default)
from crewai import Crew
sequential_crew = Crew(
agents=[researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
verbose=True
)
# Hierarchical Crew
hierarchical_crew = Crew(
agents=[researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
process="hierarchical",
manager_llm=openai_llm, # Required for hierarchical process
verbose=True
)
# Crew with Custom Manager
hierarchical_crew_custom_manager = Crew(
agents=[researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
process="hierarchical",
manager_agent=project_manager, # Custom manager agent
verbose=True
)
# Crew with Memory
crew_with_memory = Crew(
agents=[researcher, writer],
tasks=[research_task, writing_task],
memory=True,
verbose=True
)
# Crew with Full Output
crew_with_full_output = Crew(
agents=[researcher, writer, editor],
tasks=[research_task, writing_task, editing_task],
full_output=True # Return outputs from all tasks
)
# Crew with Callbacks
def task_complete_callback(task_output):
print(f"Task completed: {task_output.description}")
return task_output
crew_with_callbacks = Crew(
agents=[researcher, writer],
tasks=[research_task, writing_task],
task_callback=task_complete_callback
)
Execution Patterns
# Basic Kickoff
result = crew.kickoff()
print(result.raw) # Raw text output
# Kickoff with Inputs
result = crew.kickoff(inputs={"topic": "Quantum Computing"})
# Kickoff for Multiple Inputs
inputs_array = [
{"topic": "Quantum Computing"},
{"topic": "Machine Learning"}
]
results = crew.kickoff_for_each(inputs=inputs_array)
# Asynchronous Kickoff
import asyncio
async def run_crew():
result = await crew.kickoff_async(inputs={"topic": "Quantum Computing"})
return result
result = asyncio.run(run_crew())
# Accessing Structured Output
if result.pydantic:
print(f"Title: {result.pydantic.title}")
print(f"Key findings: {result.pydantic.key_findings}")
# Accessing All Task Outputs (with full_output=True)
for task_output in result.tasks_output:
print(f"Task: {task_output.description}")
print(f"Output: {task_output.raw}")
# Checking Token Usage
print(f"Token usage: {result.token_usage}")
Common Tool Patterns
# Search Tool
from crewai_tools import SerperDevTool
search_tool = SerperDevTool()
# Web Scraping
from crewai_tools import ScrapeWebsiteTool
scrape_tool = ScrapeWebsiteTool()
# Document Search Tools
from crewai_tools import PDFSearchTool, CSVSearchTool
pdf_tool = PDFSearchTool(pdf_path="data/research.pdf")
csv_tool = CSVSearchTool(csv_path="data/statistics.csv")
# Code Interpreter
from crewai_tools import CodeInterpreterTool
code_tool = CodeInterpreterTool()
# Custom Tool using Decorator
from crewai.tools import tool
@tool
def calculate_roi(investment: float, return_value: float) -> str:
"""
Calculate Return on Investment (ROI)
Args:
investment: Initial investment amount
return_value: Value returned from the investment
Returns:
ROI as a percentage
"""
roi = (return_value - investment) / investment * 100
return f"ROI: {roi:.2f}%"
Troubleshooting Checklist
Issue |
Potential Cause |
Solution |
Agent not using tools |
Tools not properly defined or not clearly needed in task |
Be more explicit in task description about using tools; check tool setup |
Tasks executed out of order |
Task dependencies not properly defined |
Ensure context is properly set for dependent tasks |
API key errors |
Missing or incorrect environment variables |
Check .env file and verify API keys are set correctly |
Poor agent performance |
Unclear role, goal, or backstory |
Make agent definitions more specific and detailed |
Timeout errors |
Tasks too complex or API rate limits |
Break into smaller tasks; implement max_rpm or retry logic |
Hierarchical process failing |
Missing manager_llm or poorly defined manager |
Ensure manager_llm is set or provide a well-defined manager_agent |
Tool execution errors |
Missing dependencies or incorrect usage |
Check tool documentation and required dependencies |
8. Best Practices
These best practices will help you build more effective and efficient CrewAI applications.
Agent Design
Define Specialized Roles
Create agents with distinct, non-overlapping roles for clearer responsibility distribution. Each agent should have a specific area of expertise.
Example: Instead of having two general "researchers," create a "Data Analyst" focusing on statistics and a "Industry Expert" focusing on domain knowledge.
Craft Actionable Goals
Create specific, measurable goals that guide agent behavior. Vague goals lead to unfocused actions.
Good: "Identify the top 5 trends in renewable energy with supporting data points"
Not as good: "Research renewable energy trends"
Develop Detailed Backstories
Rich backstories provide context for decision-making and shape agent personality.
Good: "You're a financial analyst with 15 years of experience in startup valuation. You've worked with VC firms and have helped assess over 200 startups. You're particularly skilled at identifying financial risks and market opportunities."
Not as good: "You're a financial expert who understands startups."
Task Design
Create Granular Tasks
Break complex processes into smaller, focused tasks for better management and performance.
Instead of: One task for "Research and write a complete report on renewable energy"
Consider: Separate tasks for research, analysis, outline creation, drafting, and editing.
Provide Clear Instructions
Include specific formats, requirements, and examples in task descriptions.
Good: "Create a 5-section report with an executive summary, market overview, competitor analysis, opportunities, and recommendations. Each section should be 300-500 words and include specific examples."
Not as good: "Write a comprehensive report on the topic."
Design Logical Task Flows
Create dependencies that mimic natural information flow and decision processes.
Example flow: Market Research → Technical Feasibility → Financial Projections → Business Plan → Executive Summary
Crew Orchestration
Start Simple, Then Expand
Begin with 2-3 agents and a sequential process, then gradually add complexity.
Iterative approach: Start with researcher + writer; once working well, add editor and refiner roles.
Equip Agents with Appropriate Tools
Match tools to agent specializations and task requirements.
Example: Equip research agents with search and web scraping tools; financial analysts with calculation tools; writers with content generation tools.
Implement Error Handling
Use guardrails and validation to ensure quality outputs.
Example: Add validation for research outputs to ensure they include proper citations; implement length checks for content; verify that reports include all required sections.
Performance Optimization
Use Appropriate LLMs
Match LLM capabilities to agent requirements and complexity.
Strategy: Use more powerful models (like GPT-4) for complex reasoning tasks and manager roles; use faster models for simpler tasks.
Implement Caching
Enable caching to reduce redundant tool executions and improve performance.
Example: Enable cache at both the crew and agent levels, especially for search operations and data retrieval that might be repeated.
Use Asynchronous Execution
Leverage async capabilities for independent tasks to improve throughput.
Example: Mark independent research tasks as async_execution=True and use kickoff_async() for parallel processing.
Key Takeaway: The most effective CrewAI applications are those that thoughtfully distribute specialized work across agents with clear goals, logical task dependencies, and appropriate tools. Start simple and iterate toward more complex systems as you gain experience.
9. Additional Resources
Continue your learning journey with these valuable CrewAI resources.
Official Resources
Community Resources
Related Technologies