Skip to content

Streaming data #2565

Answered by geezz07
geezz07 asked this question in Q&A
Jul 2, 2024 · 1 comments · 7 replies
Discussion options

You must be logged in to vote

It is the same stream object being passed to multiple crawlers via userData. Wasn't sure how else to pass the stream object to the routers. I'm now trying something like this...

let _router: RouterHandler<CheerioCrawlingContext<any, any>>;

export const initRouter = (writer: WritableStreamDefaultWriter<string>) => {
    if (_router== undefined) {
        _router= createCheerioRouter();
    }

    // Metro, Food Basics
    _router.addHandler('DETAIL', async ({ request, $, log, pushData }) => {
        ...

        if (writer) {
            await writer.write(JSON.stringify(product));
        }
    });


    // This is a fallback route which will handle the start URL
    // as well as the L…

Replies: 1 comment 7 replies

Comment options

You must be logged in to vote
7 replies
@B4nan
Comment options

@B4nan
Comment options

@B4nan
Comment options

@geezz07
Comment options

Answer selected by geezz07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants