Miroir is a web-scraper/spider allowing you to backup (‘mirror’) websites for offline use. This could prove useful if you want to backup a blog or keep a copy of a website that may be shut down. A relatively large variety of engine options are made available, such as:
Spider options: alter the ways robots.txt files are treated, parse JAVA files, accept cookies, and attempt to parse links even in javascript or other tags.
Limit options: set the maximum mirroring depth (both internal and external), the maximum size of non-HTML or HTML files that are to be mirrored, the maximum size of the mirror, the maximum time spent mirroring, the maximum transfer rate, the maximum connections/second, and the maximum number of links.
Download options: set the maximum number of connections, the timeout length, the number of retries (when the app fails to download a file), and the minimum transfer rate.
Identity options: use a proxy, set up the default referrer URL, and set up HTTP headers.
Build options: create log files, use a cache, alter the structure of the resulting copy, and make an index (websites can be downloaded ‘as is’ or can also be encapsulated in document files).
Chosen mirroring options can be saved as presets.
Several websites can be downloaded at the same time, if you tell the app to use a cache it continues where it left off.
What's new in version 1.0
Miroir is a web-scraper/spider allowing you to backup ('mirror') websites for offline use. This could prove useful if you want to backup a blog or keep a copy of a website that may be shut down. A rel
Miroir is a web-scraper/spider allowing you to backup ('mirror') websites for offline use. This could prove useful if you want to backup a blog or keep a copy of a website that may be shut down. A relatively large variety of engine options are made available, such as:
Spider options: alter the ways robots.txt files are treated, parse JAVA files, accept cookies, and attempt to parse links even in javascript or other tags.
Limit options: set the maximum mirroring depth (both internal and external), the maximum size of non-HTML or HTML files that are to be mirrored, the maximum size of the mirror, the maximum time spent mirroring, the maximum transfer rate, the maximum connections/second, and the maximum number of links.
Download options: set the maximum number of connections, the timeout length, the number of retries (when the app fails to download a file), and the minimum transfer rate.
Identity options: use a proxy, set up the default referrer URL, and set up HTTP headers.
Build options: create log files, use a cache, alter the structure of the resulting copy, and make an index (websites can be downloaded 'as is' or can also be encapsulated in document files).
Chosen mirroring options can be saved as presets.
Several websites can be downloaded at the same time, if you tell the app to use a cache it continues where it left off.