How were analytics gathered on software built for retrocomputing platforms?

Question

In modern software development, it is a common practice to include some kind of analytics tracking software to see how customers are using your software, what a typical user flow of the application is, and if any issues with the software are encountered. This luxury was not always available, especially for software on systems not connected to or predating the internet.

How were analytics gathered on legacy applications? Did software development companies use tools like registration and surveys to gather information from their customers in order to gather information on how to improve their software?

Answer 1

When I was programming BtW ("Before the WWW" - the Internet was around, but not many companies used it for much other than FTP and email), my main development tools were Turbo Pascal, Borland Pascal, and Borland C++ - yes, we were a Borland house.

When we encountered a bug in the IDE (maybe it crashed, or misbehaved, or compiled incorrect code), we'd send an email off to Borland to inform them so that they could fix it. We were usually met with either silence, or worse, request for payment for "assisted support" - which we had to explain was not the reason for the email!

It got so bad that I turned to Windows Developers' Journal (WDJ), a programmers' magazine that ran a monthly column "Bug++ of the Month". You told them of the issue, and if they could repeat it they'd contact Borland for you - oh, and printed an entry in that month's column! Needless to say, Borland was much more responsive to them than us!

When we moved to Microsoft Visual C++, we encountered different bugs - but the same stonewall from Microsoft. So I just continued submitting bugs to WDJ. Sadly, that fine publication was subsumed into Dr Dobbs Journal - who luckily have some of the articles archived!

Answer 2

As you point out, before the Internet was widely available, there was no easy way for software to “report back” on its use to the software publisher.

Publishers had a variety of ways of finding out what their users wanted though...

• Most mainstream software packages included a registration card, and users were strongly encouraged to register. Incentives included promises of cheaper upgrades; in some cases “cheaper” even meant “for the cost of shipping” (the last upgrade to Lotus Improv was one such upgrade). One real benefit of registering was that publishers would sometimes send upgrade floppies (or tapes) to fix severe bugs. None of this gave publishers any feedback, but sometimes publishers would send their registered users surveys (that was rare) or invite those that lived close by to visit the publisher and give their feedback on upcoming releases.

• The press had a huge role here. John Burger’s answer shows one example: users would write to magazines with their problems, and in some cases those made it back to the publishers’ ears. Journalists were also regularly invited to events to see upcoming releases, and give feedback. Reviews in magazines had a strong influence on the features available in software; this was heightened by the development of comparative articles (PC Magazine in particular used to compare large amounts of software and hardware, e.g. all word processors). This wasn’t necessarily all good for users though: it led to increased effort spent on things journalists care about (hence word count features in word processors, and installers — many reviews described the installer at length, even though most users would see it very infrequently).

• Some publishers had their own magazines, which users would write to — examples include Atari’s I/O in the UK, Borland’s Turbo Technix, Microsoft’s Systems Journal...

• User groups provided feedback to publishers too. Publishers paid attention at least to the larger user groups; many of these had their own newsletters or magazines which published users’ letters and problems, and of course users and publishers’ representatives would mingle at the groups’ meetings.

• Users would also write or phone publishers, and not just for technical support. This was true for any size of software publishers, although I dare say a letter or phone call would have more of an impact with a smaller publisher!

• Many publishers organised beta testing campaigns, which involved end users (as well as journalists, developers of related software...) — in fact they still do. John Gruber mentions the BBEdit beta-testing campaigns, prompted by Brent Simmons' mention of the Frontier Aretha discussion groups; one very famous beta-testing campaign is Windows 95's (code-named Chicago), which was quite wide-spread (admittedly that might be too recent to qualify as retro). Before the Internet became widely available, these would typically involve people testing pre-release software and filling in questionnaires.

As far as actual analytics goes, not having the Internet doesn’t mean they can’t be gathered. I worked on a large bespoke system once which stored analytics in its database; the analytics were dumped and returned to the publisher as part of the upgrade process. For home computers though, there wouldn’t have been much storage space to save analytics anyway...