5-4 Web Scraping Challenege

Contents

5-4 Web Scraping Challenege#

Write a program to input a keyword like “tires” and then it will search https://syracuse.craigslist.org/search/ and create a dataframe with the following columns: title, price, location, and link.

The program should scrape the first page of the search results and create a dataframe with the results.

Approach#

We are going to need to make the scraper independent of the user interface

craigslist_scraper.py

  • def scrape_craigslist(search:str) -> list[dict] will input the search term and return a list of dictionaries with the results

  • __name__ == "__main__" will use input() to take the search term and then save the json file as {search}.json or output a JSON string via print()

craigslist_scraper_ui.py

  • will be a streamlit that inputs the search term

  • calls the run_python_script function script to execute craigslist_scraper.py with the search term

  • reads the json fil or json string, deserializes, and displays the results in a dataframe

Contents