A newsgroup is a repository, usually within the Usenet system, for messages posted from many users at different locations. The term is somewhat confusing, because it is usually a discussion group. Newsgroups are technically distinct from, but functionally similar to, discussion forums on the World Wide Web.
Newsgroups are often arranged into hierarchies, theoretically making it simpler to find related groups. The term top-level hierarchy refers to the hierarchy defined by the prefix prior to the first dot.
The most commonly known hierachies are the usenet hierarchies. So for instance newsgroup rec.arts.sf.starwars.games would be in the rec.* top-level usenet hierarchy, where the asterisk (*) is defined as a wildcard character. There were seven original major hierarchies of usenet newsgroups, known as the "Big 7":
- comp.*—Discussion of computer-related topics
- news.*—Discussion of Usenet itself
- sci.*—Discussion of scientific subjects
- rec.*—Discussion of recreational activities (e.g. games and hobbies)
- soc.*—Socialising and discussion of social issues.
- talk.*— Discussion of contentious issues such as religion and politics.
- misc.*—Miscellaneous discussion—anything which doesn't fit in the other hierarchies.
These were all created in the Great Renaming of 1986–1987, prior to which all of these newsgroups were in the net.* hierarchy. At that time there was a great controversy over what newsgroups should be allowed. Among those that the usenet cabal (who effectively ran the Big 7 at the time) did not allow were those concerning recipes, drugs, and sex.
This resulted in the creation of an alt.* (short for "alternative") usenet hierarchy where these groups would be allowed. Over time the laxness of rules on newsgroup creation in alt.* compared to Big 7 meant that many new topics that could, given time, gain enough popularity to get a Big 7 newsgroup had newsgroups instead created in alt.*. This resulted in a rapid growth of alt.* which continues to this day. Due to the anarchic nature with which the groups sprung up, some jokingly referred to ALT standing for "Anarchists, Lunatics and Terrorists"
Around 1990, humanities.* was created for the discussion of the humanities (e.g. literature, philosophy), and the Big 7 became the Big 8.
The alt.* hierarchy has discussion of all kinds of topics, and many hierarchies for discussion specific to a particular geographical area or in a language other than English.
Before a new Big 8 newsgroup can be created, it must be discussed in the newsgroup news.groups, and it must be voted on(anyone is allowed to vote). The vote will only pass if at least two-thirds of all votes cast are in favour and there are 100 more votes in favour than against. Creating a new group in the alt.* hierarchy is not subject to such strict rules, but it should be discussed in alt.config first.
Types of newsgroups
Typically, a newsgroup is focused on a particular topic such as "shellfish". Some newsgroups allow the posting of messages on a wide variety of themes, regarding anything a member chooses to discuss as on-topic, while others keep more strictly to their particular subject, frowning on off-topic postings. The news admin (the administrator of a news server) decides how long articles are kept before being expired (deleted from the server). Usually they will be kept for one or two weeks, but some admins keep articles in local or technical newsgroups around longer than articles in other newsgroups.
Newsgroups generally come in either of two types, binary or text. There is no technical difference between the two, but the naming differentiation allows users and servers with limited facilities the ability to minimize network bandwidth usage. Generally, Usenet conventions and rules are enacted with the primary intention of minimizing the overall amount of network traffic and resource usage.
Newsgroups are much like the public message boards on old bulletin board systems. For those readers not familiar with this concept, envision an electronic version of the corkboard in the entrance of your local grocery store.
Newsgroups frequently become cliquish and are subject to sporadic flame wars and trolling, but they can also be a valuable source of information, support and friendship, bringing people who are interested in specific subjects together from around the world.
There are currently well over 100,000 Usenet newsgroups, but only 20,000 or so of those are active. Newsgroups vary in popularity, with some newsgroups only getting a few posts a month while others get several hundred (and in a few cases several thousand) messages a day.
A website called DejaNews began archiving Usenet and also provided a searchable web interface to it in the 1990s. Google bought the archive from them and also made efforts to buy other Usenet archives to attempt to create a complete archive of Usenet newsgroups and postings from its early beginnings. Google also has a web search interface to the archive and also allows newsgroup posting.
Non-Usenet newsgroups are possible and do occur, as private individuals or organizations set up their own nntp servers. Examples include the newsgroups Microsoft run to allow peer-to-peer support of their products and those at news://news.grc.com.
How Newsgroups work
Newsgroup servers are hosted by various organizations and institutions. Most Internet Service Providers host their own News Server, or rent access to one, for their subscribers. There are also a number of companies who sell access to premium news servers.
Every host of a news server maintains agreements with other news servers to regularly synchronize. In this way news servers form a network. When a user posts to one news server, the message is stored locally. That server then shares the message with the servers that are connected to it if both carry the newsgroup, and from those servers to servers that they are connected to, and so on. For newsgroups that are not widely carried, sometimes a carrier group is used as a crosspost to aide distribution. This is typically only useful for groups that have been removed or newer alt.* groups. Crossposts between hierarchies, outside of the big eight and alt, are prone to failure.
While Newsgroups were not created with the intention of distributing binary files, they have proven to be quite effective for this. Due to the way they work, a file uploaded once will be spread and can then be downloaded by an unlimited number of users. More useful is the fact that every user is drawing on the bandwidth of their own news server. This fact means that opposed to a point-to-point technology, the user's download speed is under their own control, as opposed to under the willingness of other people to share files. In fact this is another benefit of Newsgroups: it is usually not expected that users share. If every user makes uploads then the servers would be flooded, thus it is acceptable and often encouraged for users to just leech.
Files can be attached to a post, but there is a very small limit in the size of the file which can be attached. As such people have come up with methods to encode a file into text which is posted as a post rather then attached to a post. There is a limit to how large a single post may be as well, so methods were developed to chain multiple posts together. The News Client (the software used to read Newsgroups) then intelligently joins the posts and decodes it into a binary file.
There are two main issues that pose problems for transmitting binary files over Newsgroups. The first is Completion Rates and the other is Retention Rates. The business of premium News Servers is generated primarily on their ability to offer superior Completion and Retention Rates, as well as their ability to offer very fast connections to users. Completion rates are significant when users wish to download large files that are split into pieces; if any one piece is missing, it is impossible to successfully download and reassemble the desired file.
A number of websites exist for the purpose of keeping an index of the files posted to binary Newsgroups.