—
Notes:
- Works when directory listings (autoindex) are enabled on the folders you crawl.
- Treats links without a file extension as directories (even if no trailing slash).
- Follows http↔https on the same hostname.
- Display names are decoded and humanized (no
%20), with Title-Case.