Scrapy outputs blank history.csv file

[Rolstenhouse]

When running the command scrapy crawl yth_spider -o history.csv -L ERROR the history.csv file outputted is blank.

Cookie was set properly; and unfortunately no error logs are printed to the console.

[Luc-Olsthoorn]

Bump ^

[dwhickox]

I am also having this issue, i tried running python3 -m scrapy crawl yth_spider and the output was pretty normal except for this: 2020-07-29 15:40:05 [scrapy.core.engine] INFO: Spider opened
2020-07-29 15:40:05 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2020-07-29 15:40:05 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2020-07-29 15:53:54 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.youtube.com/feed/history> (referer: None)
2020-07-29 15:53:54 [scrapy.core.engine] INFO: Closing spider (finished)

I inputted the cookie by just copy-pasting the entirety from the request header, and if I did not do this it would give me a "not logged in on first page" error but when the cookie was correct (I think) it just didn't scrape anything.

[Imakestuff123]

I also have the same issue as dwhickox

[derectoverso]

Hey everyone, seems like there maybe was a change to the page structure or to how the code is handled on Youtube's side.

I will try to give a look at this later this week and see if I can fix this either by simply fixing the changes or by rewriting part of the scraper.

[Luc-Olsthoorn]

That would be great! I built a visualizer off of this, and just as I was about to release it, this (an obvious key component) broke. No worries though, apis change etc etc, but I would still love to release it sometime soon!

Keep us posted on your progress!

[nfsergiu]

@derectoverso have you got a chance to look into this?

It'd be greatly appreciated if you could fix the changes.

[vlad-cool]

+++

[xpresserversllc]

I am also having the same issue. Any update on this?

[wofka72]

I have same problem as this project used to solve. And I found simple solution in this article.

As I see from Acrhival Notice YouTube API V3 gives watch history.
Also you could just download it using Google Takeout. It's intuitively and there is simple instruction in the article, but I just want to say, that you could choose JSON format for YouTube Watch History and not to parse HTML manually.

Hope it will help someone.

[vlad-cool]

I have same problem as this project used to solve. And I found simple solution in this article.

As I see from Acrhival Notice YouTube API V3 gives watch history. Also you could just download it using Google Takeout. It's intuitively and there is simple instruction in the article, but I just want to say, that you could choose JSON format for YouTube Watch History and not to parse HTML manually.

Hope it will help someone.

Infortunatly, google takeout shows only history of half of year((. Maybe there is count limit of videos (I watch a few tens of them every day)

[wofka72]

I have same problem as this project used to solve. And I found simple solution in this article.
As I see from Acrhival Notice YouTube API V3 gives watch history. Also you could just download it using Google Takeout. It's intuitively and there is simple instruction in the article, but I just want to say, that you could choose JSON format for YouTube Watch History and not to parse HTML manually.
Hope it will help someone.

Infortunatly, google takeout shows only history of half of year((. Maybe there is count limit of videos (I watch a few tens of them every day)

It sounds interesting. For one of my accounts it shows full history (11200 viewed videos during 2017-2021), for the second it shows search history from 2012 (i think it's full) and video history (5400 viewed videos) only from 2016. I'd expect it shows history from 2012 for the second account.