About Project

Friday 26. October 2018 Posted by: oshane-bailey NodeJS Web Scrapping
Versatile web scraping utility designed with collecting time series data in mind.

Install

npm i scrappi

Usage

Main use case of scrappi would be to scrape a webpage for value(s), post to an api endpoint, at a specific interval

import scrappi from 'scrappi'


let result = await scrappi({
    target: 'https://google.com', //Get html
    endpoint: 'https://coolapi/post', //Post results to here
    tick: 500, // Every 500ms
    onDocumentReceived: (html) => {
        //do whatever with html in here
    },
    post: (json) => {
        //send json to endpoint
    }
})