Scrape Latest 20 TechCrunch Articles

For TechCrunch, this automated workflow scrapes the latest 20 articles, extracting key details like titles, URLs, images, and publication dates. It streamlines content collection, enabling users to stay updated with minimal effort.

7/8/2025
9 nodes
Medium
manualmediumsplitoutapiintegration
Categories:
Manual TriggeredMedium Workflow
Integrations:
SplitOut

Target Audience

This workflow is ideal for:
- Content Creators: Bloggers and journalists looking to gather the latest tech news efficiently.
- Marketing Professionals: Those who want to stay updated with industry trends and insights for strategic planning.
- Developers: Individuals interested in automating data collection from TechCrunch for analysis or integration into other applications.
- Researchers: Academics or analysts studying trends in technology and startups.

Problem Solved

This workflow addresses the challenge of manually tracking and collecting the latest articles from TechCrunch. By automating the scraping process, users save valuable time and ensure they receive the most recent updates without the need for constant manual checking.

Workflow Steps

  • Manual Trigger: The workflow begins when the user clicks ‘Test workflow’.
    2. Request Latest Articles: It sends an HTTP request to TechCrunch to fetch the latest articles.
    3. Parse Articles Box: The HTML response is parsed to locate the section containing the articles.
    4. Extract Posts: All individual posts are extracted from the parsed HTML.
    5. Split Out Posts: Each post is split out for further processing.
    6. Parse Each Post: For each individual post, details such as image, title, URL, and creation date are extracted.
    7. Request Post Details: A separate request is made to fetch the detailed content of each post.
    8. Parse Content and Metadata: The content, title, thumbnail, and creation date are parsed from the detailed post page.
    9. Save Values: Finally, the extracted values are saved for further use or analysis.

    • Customization Guide

    Users can customize this workflow by:
    - Changing the Source URL: Modify the URL in the 'Request TechCrunch Latest Page' node to target different sections of TechCrunch or other websites.
    - Adjusting CSS Selectors: Update the CSS selectors in the parsing nodes to extract different data elements or additional details from the articles.
    - Modifying Output: Alter the 'Save the values' node to change the structure of the saved data or to include/exclude certain fields.
    - Adding Additional Nodes: Integrate more nodes for further processing, such as sending the data to a database, email, or another API for analysis.