**Note:**Please be aware that due to recent security updates implemented by Twitter, the code might encounter issues or fail to run as expected.
This project is a Twitter scraper tool designed to gather user IDs, names, and profile links based on specified keywords. Developed as a portfolio project, it offers insights into Selenium and data scraping techniques, fostering a comprehensive understanding of web scraping methodologies.
The primary goal of this project was to delve into Selenium and data scraping, aiming to achieve an in-depth comprehension of scraping methodologies. Through the development of this Twitter scraper, I attained valuable experience in navigating the complexities of web scraping, particularly in dynamically changing environments like Twitter.
While developing the scraper, one significant challenge was the dynamic nature of Twitter's DOM elements. Constant changes in the structure made it difficult to locate specific elements consistently across various searches. However, I successfully overcame this hurdle by devising strategies to dynamically locate elements, enhancing the script's adaptability.
In the future, I plan to extend the functionality by adding a profile scraper feature. This enhancement will enable the tool to extract comprehensive details from user profiles, enhancing the scope and utility of the application. Additionally, due to Twitter's frequent updates and changes in its structure, the effectiveness of this project might be affected. Continuous updates and adaptations will be required to ensure the tool's consistent functionality.
Clone the repository
git clone https://github.com/Ja-yy/Twitter-scraper-streamlitpipenv shellpipenv installstreamlit run app.pyRemember, while this scraper might unravel Twitter mysteries, it won't fetch you coffee—yet! Keep coding, and who knows what the next project might brew!!