Fix YouTube Music search

[TobiGr]

• I carefully read the contribution guidelines and agree to them.
• I have tested the API against NewPipe.

YouTube changed the JSON structure

Fixes TeamNewPipe/NewPipe#5191
@badprogramshere Can you test the APk below and tell me if you spot any bugs?

app-debug.zip

[badprogramshere]

Sorry about the delay! I've just tested it. I can search albums correctly and play most tracks as usual, but the album content seems to include music videos with the normal YouTube Music tracks. This isn't too much of a problem for the most part though, and this is the same content from the desktop website, so I don't think this can be fixed.

[TobiGr]

I noticed that, too. However, that is the result YouTube Music also uses on their website. So I cannot do much about it.

[badprogramshere]

Yeah, I can understand that. Thanks for the patch anyway! I can tolerate having to use the music-video versions of songs.

[TobiGr]

You should get the YouTube Music version of the videos now. app-debug.zip

@TiA4f8R I don't know what you mean. I could not get this data. Did you inspect the website to get that JSON?

[TobiGr]

ah ok. YouTube provides a different JSON to the browser version. I cannot use that info in this case.

Try that: app-debug.zip

[AudricV]

@TobiGr It will be great if this fix is included in 0.20.7.

[TobiGr]

Added to the to do for 0.20.7.
However, I cannot reproduce your json structure.
Can give me clear steps what to search and which filters to use to get a your file containing a toggledServiceEndpoint?

A separate questions is whether the playlist id allows tracking

[AudricV]

@TobiGr First about tracking, there's no tracking in the playlist itself if you use this endpoint. The "clickTrackingParams" is only used in the website or in YT Music's app, that you can see many times in the JSON response.
About giving you clear steps, I will first try something on my local side and let you know if that I did works.

[AudricV]

@TobiGr So can you do the contrary?

[TobiGr]

@TiA4f8R see #484 (comment)

[TobiGr]

ah i see. our test cases are bad.
Thanks for your patience!

[B0pol]

Exception

• User Action: requested playlist
• Request: https://music.youtube.com/playlist?list=RDCLAK5uy_l5CaXPtlHAWKUBF5rj9w5xII1Bugs-iwc
• Content Country: FR
• Content Language: fr-FR
• App Language: fr_FR
• Service: YouTube
• Version: 0.20.6
• OS: Linux Android 9 - 28

Crash log

org.schabi.newpipe.extractor.exceptions.ParsingException: Got HTML document, expected JSON response (latest url was: "https://www.youtube.com/watch?v=CLAK5uy_l5C")
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getValidJsonResponseBody(YoutubeParsingHelper.java:575)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getResponse(YoutubeParsingHelper.java:588)
	at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeMixPlaylistExtractor.onFetchPage(YoutubeMixPlaylistExtractor.java:59)
	at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:54)
	at org.schabi.newpipe.extractor.playlist.PlaylistInfo.getInfo(PlaylistInfo.java:30)
	at org.schabi.newpipe.util.ExtractorHelper.lambda$getPlaylistInfo$9(ExtractorHelper.java:172)
	at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$RLOrwix0kCDfcc_WpFBc8DTJw0Q.call(Unknown Source:4)
	at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
	at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5330)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
	at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15753)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
	at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5330)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
	at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:614)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:764)

---

[B0pol]

Minor changes, you can merge after this