Sebastian Nagel · @sebnagel
4 followers · 4 posts · Server fosstodon.orgReleased #CrawlerCommons 1.4: Java 11, #RobotsTxt compliant with #rfc9309 - https://github.com/crawler-commons/crawler-commons#18th-july-2023----crawler-commons-14-released
#crawlercommons #robotstxt #rfc9309
Last updated 1 year ago