
Imminent 'todo' list
====================

These should be done 'fairly' soon:

* check operation with ArcWeb
* socket watcher support
* option to download all html before downloading images
* config file parser to cope with spaces at beginning of line
* aborting fetches should leave entries in the logfile
* option for non-html logfile
* fetch file:/ styles url's.
* why does http://www.isi.edu/in-notes/iana/assignments/media-types/application/zip
  not get fetched if set to exclude *.zip?
* option to only check for existance of pages linked to.
* option to check links, following one link offsite.

Slightly longer term:

* Password protected sites
* option(?) to fetch local images/links before offsite ones.
* reduce icon flicker slightly by not blanking icons unless we need to?
* Progress indicator for rewrite
* Allow rules to be added whilst a fetch is in progress
  (eg. skip all pages/images/whatever on a certain site)
* Allow fetches to be saved in other places, probably packaged
  up inside a little application along with the logs and things.
* handle http temporary and permanent redirections properly
* download of java applets and required classes


[See !Help file for full todo list. I'll probably only implement
things off that list when (if??) the above list is empty!]

If anyone does a VIX module for this, can they let me know and I'll
include it with WebGet. Although I have the VIXpacker, I don't
have Voyager and don't really understand what's involved.

There are now two WebGet mailing lists - please see the !Help file
to find out how to subscribe; most people will find it worthfile
to join the announcements list!

Summary of changes
==================

0.18		Released : 25/03/01
11/03/01	Replaced built in resolver code with code to call the
		  Resolver module (or the variant of it running).
09/10/00	Now sends Accept: header in the HTTP request, to allow
		  it to work with some MS webservers.
11/02/00	Fixed some bugs with resolving of URLs, and stopped
		  webget changing mailto: urls.
08/02/00	Added option not to run logfile at the end of a fetch
		  ('show-log' in the choices file.)

0.17		Released : 22/08/99
22/08/99	Now shows 'Rewriting' or 'Fetch complete' as appropriate.
21/08/99	Fix for URL resolving going wrong with people did relative
		  links beginning http:, which the RFC forbids.
		Another set of new animations
		Fixed temporary / permanent relocations so they resolve
		  correctly.
		Fixed the 'Open root' / 'Launch URLs' menus getting corrupted
		  when a new fetch was added.
		Fix for Content-Type: text/html; something else not getting
		  recognised as html.
06/10/98	Slightly changed (but not much better) animation
		  sprites.
19/10/98	Logs are now in <WebGet$WriteDir>.Logs
		  (won't affect most people)
03/11/98	Fixed a problem or two parsing <meta http-equiv=
		  refresh url=... > tags.
06/12/98	Fixed problem where HTTP would be treated different
		  to http.
		Fixed another code path to remove control characters
		  from a URL.
13/12/98	Ignores links to javascript functions when looking
		  for URLs to fetch.
10/2/99		Fixed flex memory error whilst idle when max_fetchers
		  was at the maximum value.
		Choices file is now passed through OS_GSTrans - this
		  means system variables may be used - eg:
		  http_proxy: <NCFresco$ConfigProxyHttp>

0.16		Released : 04/10/98
26/09/98	Fixed problem with images not getting fetched
28/09/98	Loading of the mimemap module delayed until the
		  start of a fetch, so webget can be loaded
		  before the internet suite.
04/09/98	New versions of ModInst and mimemap module,
		  to fix crashes under RiscOS 3.1.
		  (0.03 and 0.10 release 2, respectively)
04/09/98	HTML of the form <a href = "..> is coped with
		  (the spaces around the = where causing the parser
		  to fail previously)

0.15		Released : 16/09/98
02/09/98	Replaced internal mimemap code with calls to the
		  mimemap module.
		Fixed bug where total shown in the status window
		   would be too small when the total got over 3M.
15/09/98	Added 'Launch URL' and 'Open root' options to icon
		  bar menu.
16/09/98	Added !ModInst + MimeMapper module to distribution.

0.14y		Released : 27/09/98
27/09/98	Fixed bug where images weren't fetched

0.14		Released : 30/08/98
25/08/98	Links to pages that haven't been fetched are now
		  rewritten to point at the original http/ftp link.
		Log file display should now work with ArcWeb
30/08/98	Fetching order is now far more structured - when
		  deciding what to fetch next, things closer to
		  the root page are preferred, and within that subset
		  html is preferred over inline images.
		More control over fetching of inline images
		

0.13		Released : 23/08/98
18/08/98	Fairly brutal rewrite of HTML parser, is now more
		  tolerant of broken HTML (eg. missing "s)
		Stopped updating of icons when their content hasn't
		  changed (reduces flicker in status win)
		Log file now says 'No errors' if there weren't any
		  errors during the fetch.
22/08/98	Dragging of urls & URI files to the icon bar /
		  addurl window added.
23/08/98	'Cannot resolve' message now includes hostname
		Prefix used for local URLs now configurable so
		  WebsterXL can view fetch pages. See Choices file.
		Fixed bug in HTML parser where <a\nhref="...">
		  wouldn't be recognised as a link.


0.12		Released : 16/08/98
10/08/98	WebGet now follows / rewrites <META...> refresh tags.
		Defaults for include/exclude now put in Add URL
		  dialogue box.
		Follows links in client side image maps
		Fixed minor bug in html parser where it'd treat
		  <AWIGGLE HREF...> the same as <A HREF=...>
		Fixed bug where links=dir didn't work right when
		  a url was specified as http://host
11/08/98	Increased length of text in status window url icons,
		  and made the code trim the left end when they grow
		  above that length.
		Menu button now works over status window.
		Internal lists now cleared at end of a fetch
12/08/98	Added - and _ to the valid chars in the path section
		  of the url too (added them in the hostname before!)
		Two consecutive /'s in a filename/path no longer map
		  to .., as that's an invalid RiscOS filename!
13/08/98	Added -fetch <fetchfile|desc> and -quit command line
		  options. (NB. WebGet must not already be running
		  for these to work!)
		Status window automatically opens at the start of a
		  fetch - choices file option open_status_win: no
		  can disable this.
16/08/98	Corrected rewriting of <a href="#foo"> so it works.
		When rewriting URLs in fetched pages, everything
		  should now be relative.


0.112		Released : 09/08/98
09/08/98	Fixed addurl creating files with the wrong tags
		(include-files instead of include)


0.11		Released : 09/08/98
07/08/98	Removed 'Fetch Now' button accidently left in
		AddURL dialogue box.
08/08/98	Icons in status window are now right aligned.
09/08/98	Include / exclude options to allow the user to
		include and exclude URLs matching the given patterns.
		Fixed bug where we'd sit idle at the end of a fetch for
		  30 secs if a persistent connection was open.
		Fixed bug where adding a URL without a description saved
		  it in the front place.
		Expanded range of valid characters in filenames to
		  include - and _.
		Changed main menu stop fetch options so they're more
		  self explanatory (hopefully).
		Fixed bug where urls like http://a got saved in the
		  wrong place.
		Added versions of the animation sprites for
		  rectangular pixel modes.
		Removed raFS from the WebGet distribution.
		

0.10		Released : 04/08/98
		First public release
