YouTube Scraper

Effortlessly scrape, store and manage YouTube video data for your personal or project needs.

Problem

A YouTube video management business faces the recurring challenge of efficiently cataloging and organizing an extensive library of videos for its clients. Manually collecting and updating video information such as titles, descriptions, view counts, and upload dates can be a time-consuming and error-prone task.

Solution

This application utilizes web scraping techniques to extract and save essential YouTube video data. By using this application, the business can ensure accurate and up-to-date information for their clients, streamline their video management workflows, and ultimately provide a more seamless and efficient service.

Features

  • Register and login.
  • Add YouTube videos to a database with various details by web scraping the YouTube servers.
  • View list of videos.
  • Increase and decrease ratings.
  • Delete videos.

Utilization

  • Fully responsive.
  • Authentication with JWT.
  • Animations using Framer Motion.
  • Node.js server deployed to GCP using GCP App Engine, GCP Cloud Build, GCP Cloud Storage, GCP App Engine Admin API and GitHub Actions.
  • PostgreSQL database deployed to AWS using AWS RDS.

Automated Tests

  • Unit tests with Jest.
  • Integration tests with Jest & SuperTest.
  • End-to-End tests with Cypress.

Tools

  • HTML
  • CSS
  • JavaScript
  • React.js
  • Node.js
  • React Hook Form
  • Yup
  • Cheerio
  • PostgreSQL
  • Jest
  • SuperTest
  • Cypress
  • Netlify
  • GCP App Engine
  • GCP Cloud Build
  • GCP Cloud Storage
  • GCP App Engine Admin API
  • GitHub Actions
  • AWS RDS

Deployment

  • React.js front end deployed to Netlify.
  • Node.js server deployed to GCP.
  • PostgreSQL database deployed to AWS.

YouTube Scraper