Contact Us
Home / Blog / Mobile App

How to make a CLI based movie scrapper using NodeJS

Mobile App | July 3, 2022

How to make a CLI based movie scrapper using NodeJS

This guide will instruct you on how to make a CLI application that will ask the user for movie, and present info about that movie in return. To be most successful you should have at least a beginers understanding of Node and NPM. You should also have a basic understanding of using bash to navigate files. Although you are free to use a GUI application if you wish, only terminal examples will be provided.

Prerequsites

  • A text editor or IDE of your choice.

  • A terminal that is capable of running bash commands.

Part One (Setup Folder & File Structure, Install NPM Packages)

First, create the root folder for the project. This guide will call it movie-scrapper. Then navigate into the newly created folder. 

mkdir movie-scrapper && cd movie-scrapper

Within the movie-scrapper directory run the following command.

This will create the package.json file. You need to make two modifications to this file.
Add "type": "module" to the top level, and "start": "node src/index.js" to the scripts section.

It should look like this. (You may have some slight differences depending on your pesoral npm settings, just focus on the changes needed above) 

{"name":"movie-scrapper","version":"1.0.0","description":"","main":"index.js","type":"module","scripts":{"start":"node src/index.js","test":"echo "Error: no test specified" && exit 1"},"keywords":[],"author":"","license":"ISC","dependencies":{}}

Run this next command in your terminal to install the npm packages axios dotenv and inquirer 

npm i axios dotenv inquirer

You should now see a node-modules directory in your root folder. 

.
├── node_modules
├── package-lock.json
├── package.json

You should also see the newly installed packages listed in your package.json dependcies

Now you are going to create a .env file that will hold your API key for TheMovieDB. If you are using git or any other version control for this project, remember to not commit this file.

Replace PUT_YOUR_API_KEY_HERE with your actual API key. Keep the backslashes and quotation marks. 

echo TMDB_API_KEY="PUT_YOUR_API_KEY_HERE" >> .env

You can verify the file was created using the cat command

You will see th following output with your API key in place of the Xs

Create the src directory that will contain the logic for the program, and navigate into it. 

mkdir src && cd src

Create the the following files in the src directory:

Movie.js
config.js
getByID.js
index.js
prompt.js
searchMovie.js

You can do this in one command in your terminal

Your project should now have the following file stucture 

.
├── .env
├── node_modules
├── package-lock.json
├── package.json
└── src
    ├── Movie.js
    ├── config.js
    ├── getByID.js
    ├── index.js
    ├── prompt.js
    └── searchMovie.js

Part Two (Javascript Files)

Copy/Paste the following code to each correspodning file

src/config.js

import dotenv from "dotenv";
dotenv.config();

export default{
    tmbdkey: process.env.TMDB_API_KEY,
};

src/getByID.js

import config from "./config.js";
import axios from 'axios'

const getByID = async (id) => {
        const options={
            params: {
                api_key: config.tmbdkey,
                language: "en-US",
                append_to_response: `credits`
            }
    }

    let response = await axios.get(`https://api.themoviedb.org/3/movie/${id}`, options)
    return response.data

}

export default getByID;

src/index.js

import inquirer from 'inquirer';
import Movie from './Movie.js'
import moviePrompts from './prompt.js'

const movieResponse = await inquirer.prompt(moviePrompts)
const selectedMovie = await new Movie(movieResponse.movieID);
selectedMovie.summary()

src/Movie.js

import getByID from './getByID.js'

class Movie {
    constructor(id) {
        return (async () => {
            this.details = await getByID(id)
            return this
        })()
    }

    get title() {
        return this.details.title
    }

    get tagline() {
        return this.details.tagline
    }

    get overview() {
        return this.details.overview
    }

    get directors() {
        const directors = this.details.credits.crew.filter(
            (obj) => obj.job === 'Director',
        )
        return directors.map((director) => director.name)
    }

    get writers() {
        const writers = this.details.credits.crew.filter(
            (obj) => obj.job === 'Screenplay',
        )
        return writers.map((writer) => writer.name)
    }

    get cast() {
        const cast = this.details.credits.cast.slice(0, )
        return cast.map((castMember) => ({
            name: castMember.name,
            role: castMember.character,
        }))
    }

    summary() {
        const summary = 
`
${this.title} - ${this.tagline}
Directed By: ${this.directors}  Written By: ${this.writers}${this.overview}

Starring:
${this.cast.map(castMember => ` ${castMember.name} as ${castMember.role}`)}
`
        console.log(summary) 
    }
}

export default Movie

src/prompt.js

import searchMovie from './searchMovie.js'

const moviePrompts = [  
    {
        name: "name",
        type: "input",
        message: "Enter a movie to search:"
    },
    {
        name: "movieID",
        type: "list",
        message: "Select a movie:",
        choices: (answers) => searchMovie(answers.name)
    }
]

export default moviePrompts

src/searchMovie.js

import config from "./config.js";
import axios from 'axios'

const searchMovie = async (movie) => {
        const options={
            params: {
                api_key: config.tmbdkey,
                language: "en-US",
                query: movie,
            }
    }

    const simpleList = movieObj => ({name: `${movieObj.title} (${movieObj.release_date.slice(0,)})`, value: movieObj.id})

    const res = await axios.get('https://api.themoviedb.org/3/search/movie', options);

    const list = res.data.results.map(simpleList).slice(0,)

    return list;

}

export default searchMovie;

Part 3 (Execution)

At the root of your project run

You will be preseted with the following. 

> movie-scrapper@1.0.0 start
> node src/index.js

? Enter a movie to search:

Follow the prompts as given by the application.

This content was originally published here.