Music Analytics Scraper

This repository contains code and documentation for scraping music statistics like Spotify Music Streams from various sources.

Table of Contents

• Introduction
• Build
• Usage
• Contributing
• License

Introduction

The Music Analytics Scraper project aims to collect and analyze music statistics from different platforms and sources. By scraping data from popular music streaming services, social media platforms, and other sources, we can gain insights into music trends, popularity, and user preferences.

And, by using this data, we can create reports, charts, and other visualizations to help us understand the music industry better.

Requirements

• Docker
• Docker Compose

Usage

To usage this repository, you need to have:

• Spotify API key, you can get here
• Youtube Data API key, you can get here

With, these keys:

• Copy .env.example file and rename it to .env
• Replace the following values of the variables with your keys

SPOTIFY_API_CLIENT_ID=your_spotify_client_id
SPOTIFY_API_CLIENT_SECRET=your_spotify_client_secret
YOUTUBE_API_KEY=your_youtube_api_key

Output

The result output will be saved in a PostgreSQL database, you can access the database using the following credentials (default) (Please see the docker-compose.yml file):

host: localhost
user: postgres
password: mypassword
database: music_analytics

Data content

Current data content:

• Artist
• Album
• Track
• Artist Statistics

From the following sources:

• Spotify
• Youtube

Example of running:

Adding Artist:

make add_artist

Updating Artist ids:

make update_artist

List Artists:

make list_artists

Updating Artist Statistics:

make update_artist_stats

Updating All Artists Statistics:

make update_all_artists_stats

Skipping platforms:

Spotify:

make update_all_artists_stats --skip-spotify

Youtube:

make update_all_artists_stats --skip-youtube

Reports

Show Spotify Artist Report:

make show_spotify_report

Show Youtube Artist Report:

make show_youtube_report

Compare Artists Report:

make compare_artists_report

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.