crwlr.software
@crwlr.bsky.social
π€ 34
π₯ 152
π 14
crwlr.software is a collection of open source PHP composer packages for web crawling and scraping.
ππππππππππππππππππππππππππππππππππππππππππππππππππππππππππππ
11 days ago
0
0
1
Made it to 10,000 installs of the crwlr/crawler package! π₯³πͺπ» Hereβs to the next 10,000! π₯
3 months ago
0
0
1
Reached some install milestones for the crwlr packages π - url passed 50,000 installs - query-string is at an unbelievable 3,000,000 installs (thanks to bref!) - robots-txt is approaching 15,000 - crawler will soon reach 10,000 Thanks to everyone using and supporting the packages! π«Ά
6 months ago
0
1
1
Made it to 350 βοΈs on the crawler package Github repo! ππͺπ»π«Ά
github.com/crwlrsoft/cr...
loading . . .
GitHub - crwlrsoft/crawler: Library for Rapid (Web) Crawler and Scraper Development
Library for Rapid (Web) Crawler and Scraper Development - crwlrsoft/crawler
https://github.com/crwlrsoft/crawler
9 months ago
0
0
0
Tagged a bugfix release of the robots-txt package, thx to this issue
github.com/crwlrsoft/ro...
by Martin-Matchory. ππͺπ»
loading . . .
Doesn't seem to work with this robots.txt Β· Issue #12 Β· crwlrsoft/robots-txt
robots.txt ---------------------------- User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php User-agent: * Disallow: /*blackhole Disallow: /?blackhole ---------------------------- url...
https://github.com/crwlrsoft/robots-txt/issues/12
9 months ago
0
0
1
A new blog article is out! π€π Learn how to use a crawler to perform a health check and warm your cache after every deployment of your website or web app. Ensure smooth performance and error-free user experiences. Happy reading!
www.crwlr.software/blog/post-de...
loading . . .
Crwlr Recipes: Using a Crawler for Website Error Detection and Cache Warming
Have you ever deployed your website or web app, only to discover hours later that youβve introduced bugs or broken links? Or do you clear the cache with every deploy, leaving the first users to experi...
https://www.crwlr.software/blog/post-deployment-health-and-cache-warming-crawler
9 months ago
0
0
1
Ah, it seems like XPath is no longer an option in the new DOM API in PHP 8.4. I assume it wonβt be a problem if we remove this feature in a new major version of the crwlr/crawler library?
11 months ago
1
1
2
The new HTML5/DOM functionality in PHP 8.4 is amazing! Iβm working on adding it to crwlr/crawler to leverage it when PHP 8.4 is used. This will enable features like CSS pseudo-selectors not supported by Symfony DomCrawler.
11 months ago
0
2
1
PHP 8.4 just dropped! π₯³ππͺ
php.net/releases/8.4...
Great release page! Well done! Already added running tests on 8.4 in CI, so all crwlr packages should be ready for the upgrade! πͺπ
loading . . .
PHP 8.4 Released
PHP 8.4 is a major update of the PHP language. It contains many new features, such as property hooks, asymmetric visibility, an updated DOM API, performance improvements, bug fixes, and general cleanu...
https://php.net/releases/8.4/en.php
11 months ago
0
1
0
Hey Bluesky! π What better time to move over from the old place? So, crwlr is now here on Bluesky too! π₯³π
11 months ago
0
0
0
you reached the end!!
feeds!
log in