Internet Search: Yesterday, Today & Tomorrow

In the beginning (yesterday) the Internet was an academic network and then the Free-nets (including the National Capital FreeNet of which I was an early member and information provider) were created providing a place for community organizations and bringing the Internet to the people.

In these early days, before the World Wide Web, the Internet was primarily text based and used search tools known as Archie, Gopher, Veronica and Jughead to search for documents stored online. People also used services such as Usenet to access the equivalent of today’s web forums and IRC (Internet Relay Chat) as a group and private real time messaging service. Most importantly we all had Email, which IMHO is still the most important thing the Internet gives us as individuals.

Then came World Wide Web and HTML and everything changed. The Internet was still non-corporate being primarily educational institutions, non-profit organizations and individuals but that soon changed, many say for the worst, when corporations were allowed onto the network. I would certainly miss online banking and shopping and streaming services have given us access to non-North American “television” we would not have had otherwise.

The WWW gave individuals an opportunity to have their own place on the Internet through personal websites (also called Home Pages back then). Internet Service Providers would provide customers with web storage they could use to create their own web pages using HTML and sites like GeoCities made it even easier. Then came Myspace, a sort of Facebook lite. There were other sites serving the same user base that wanted their own place on the Internet and they all co-existed peacefully. And then came Facebook and everything changed for the worst. Most people criticize Facebook for it’s tracking of users and monetization of their and their “friends” personal information, but to me the most evil thing about it is it’s business model of trying to keep users away from the open Internet and dependent on their proprietary site.

At one time, long before Facebook, there was even a print Internet Yellow Pages that listed all the significant websites on the World Wide Web but it quickly became necessary to have some online tool for people to find what they were interested in without depending on prior knowledge, friends or just luck.

When we started using the Internet for research or to find information were not looking for specific answers to specific questions but for resources where we could find those answers.

And perhaps the best tool for that was the original Yahoo Directory which was a hierarchical listing by subject of web resources curated by librarians to ensure the legitimacy of the sources. Other directories also existed, particularly subject specific ones. As the Internet grew exponentially keeping up a complete directory became an impossible task, or at least economically impossible to compete with search engines that also existed at that time,

In the beginning we used search engines the same as way the Yahoo Directory, to find resources where we could find the information we were seeking. Perhaps the best of the early search engines and my personal preference was Digital Equipment Corporation's AltaVista search engine which allowed users to do a Boolean Search using AND, OR & NOT operators. Soon people started using search engines to find specific answers to specif questions.

Alta Vista and almost all other search engines were surpassed by the original Google search engine whose algorithm impressed everyone so much that it became the dominant search engine. It’s advanced search mode also allowed Boolean searches. It became my (and most peoples) search engine of choice for a long time.

Then came the enshittification of both search and the Internet as a whole.

The enshitification of search happened as Google gained an effective monopoly on Internet search, so much that to search the Internet became “to google” as nearly all searchers were done using Google. And then we saw the gradual degrading of Google as it monetized it’s search engine. We would see promoted links at the top of search results that were paid for. Searches for, as an example, Ford F-150 would have Chevy Silverado as the first listed result because General Motors paid for that. And then we started getting results in the form of answers to questions rather than as links and people referring to “Google said/told me” rather than referring to the sources Google found.

Somewhere along the line the advanced Boolean search capability disappeared from Google and then it became contaminated by LLM chatbots spouting spurious answers and information. It may be possible with enough effort to turn off the AI slop in Google but personally I would not trust that that is so. Google’s once famed reliability is now in the dumpster. And of course Google has become infamous for tracking it’s users.

People have started to slowly move away form Google to privacy supporting search engines like DuckDuckGo, although it has been criticized for it’s optional AI features although it is a lot easier to disable them in settings than with Google. I personally use the non-AI version of Duck Duck Go (https://noai.duckduckgo.com/) which has the AI features disabled. I only wish it had obvious Boolean search capabilities, although there are apparently ways to do Boolean searches and other advanced search techniques for DDG (that I did not know about until I researched this post).

But the enshitification goes beyond Google search and has infected the whole of the Internet/World Wide Web. Over the last 20 years or so we have seen a proliferation of fake news and disinformation sites and social media has increased the amount of misinformation and misinformation online by orders of magnitude.

But the user is also to blame. The reason for Facebook’s success is the fact that consumers today put convenience above all else and when you add the super convenient magic answer machine LLM based AI chatbots that base their answers on whatever is repeated most (the GIGO principle) the result is inevitably garbage.

Tomorrow’s search function requires a better way for those of us more interested in accuracy than convenience. Let us suggest a new model that puts a boolean search engine on top of a directory of trusted sites and builds from there.

We start with an original Yahoo type directory curated by librarians and subject specialists. The directory is hierarchical starting with broader subjects going to lower ones. One can browse or search directory to find the field of knowledge you are interested in and select relevant websites from there.

The curators will not attempt the impossible task of vetting all contents on the websites/resources but they will be selected according to the trustworthiness of those responsible. Different categories of resource will be vetted differential according to their nature.

Information resources on science, the humanities and the social sciences will be judged according to the reliability of the content as ascertained by the trustworthiness of those responsible for them.

There will be a general information category for encyclopedias and similar broad works.

Journalistic sources will be judged again according to the journalistic principles of the organizations, ethical, fact checking, distinguishing opinion from news content, etc.. Sites that are solely expressing opinions will be identified as such and where possible identified according to bias, right leaning, left leaning, etc. Satire sites will be identified as such for those that cannot figure that out.

Political sites will not be vetted according to accuracy but according to whether they are actually who they say they are and not attempts to spoof or misrepresent the opinions of politicians or political organizations. Similarly for corporate and banking sites as a protection against fraud.

Social media sites will be included in the listings for those that seek them out but will not be included automatically in searches.

The next level of search will be the ability to search not just for information resources/websites but also within them like a normal web search but restricted to sites within the directory, as a whole or by specific subject matter, or specific website.

And finally a full internet search will be available where that is desired. The ability to exclude social media sites (and perhaps certain other categories) will be included. All searches will have full Boolean search capability and resources on how to understand and use the Boolean search capability will be provided.

A final capability, which i am on the fence about whether it should be included, is a natural language question search capability with an algorithm to translate that into boolean search terms.

The big question here becomes how can this be funded. Ideally enough users would be willing to pay for accurate search to make it work, but let’s not delude ourselves about the majority of Internet users. So it would probably require some major donors willing to fund it because it is good for society, and hopeful broadly distributed, with small individual donations being at least a significant portion of the funding.

United States of Armageddon

If American current events were a dystopian TV series we would all be saying this is over the top, no one can be this diabolically cruel and evil and dementedly stupid at the same time. Let’s just hope it’s cancelled after one season.

Trump is the main cheerleader for the us vs them philosophy. Politically it is that his supporters can do nothing wrong and are worthy of pardons for the moist heinous crimes while political opponents can do no good and are evil and horrible people. More dangerously he sees Americans as normal white cis gender straight right wing Christians vs foreigners from shithole countries and perverts, a categorization that anyone who knows history can see the parallel to. And the only role of women is to mandatorily produce babies.

Those of us watching from the outside do not even know who to blame. Certainly renowned snake oil salesman Trump is at the top of the list and he is (or was) the ideal front man.

But one person does not make this all happen, it takes a movement.

At the top of the list is organized religion, call it evangelism, fundamentalism or the Christian right, but it enables it all. I have always said it is easy to get bad people to do evil but it takes religion to get good people to do evil, because it is not evil if god (or those you believe speak for him) says it is what he wants.

And then there are the tech billionaires that financed Trump’s election victory (even if it’s legitimacy is questionable). There is a conundrum here because technology is based on science and the Trump regime is, if anything, anti-science. But the tech industry has long abandoned science when they decided wasting large amounts of water and energy to perform meaningless mathematical calculations can create value or that the all of the garbage on the Internet can be used to make a software machine intelligent.

So we have current situation where there is apparently no line the fascist Trump white supremacist regime cannot cross that will enable it to be stopped.

Those of us in parliamentary democracies are amazed, while probably not amazed as we know the American system is different, but disappointed that the legislature (Congress) could just not get rid of Trump by voting non-confidence in him, assuming at least a few Republicans are sane.

Separation of powers does not seem to be providing the safeguards it is supposed to. The procedures to remove a president for misconduct or incapacity are too cumbersome to be useful. While Congress supposedly controls government finances and theoretically should be able to just refuse to fund ICE or refuse to approve Trump’s tariffs, with a one man executive who refuses to abide by the law and the constitution and a corrupt Supreme Court willing to let him do whatever he wants it seems there is no solution.

European Multi Party PR & American Two Party FPTP Electoral Systems: Built on Coalitions

The European electoral system is based on Proportional Representation (PR) with multiple parties while the American electoral system is based on the First Past The Post (also known as Single Member Plurality (SMP)) with only two parties.

Interestingly the Canadian and United Kingdom systems feature an FPTP voting system with multiple parties.

However both systems depend on coalitions to elect a government, the difference being when the coalition is formed.

In Europe you vote for the candidate or party that best represents your views and elect a legislature that represents the views of all voters, proportional to their support. The parties then have to co-operate and negotiate a government that has the support of a majority of the legislature (which reflects a majority of voters because of the Proportional Representation system). These coalition governments may be a coalition of parties on the left or right or a coalition of centre left and centre right parties. The whole system encourages co-operation amongst parties.

In the United States the party structure is made up of two, what some call “big tent parties”, one party for everyone left of centre and one for everyone right of centre, in effect two coalitions of voters. Theoretically this should result in a centre left and centre right party but obviously it does not always work that way. Although the current American political situation is not what one would call historically normal, with the Republican Party being taken over by fascists.

Interestingly, historically in Canada we had two major parties, one centre left, the Liberals and one centre right, the Progressive Conservatives (PC) until the far right took over the PC Party, even changing it’s name to the Conservative Party.

The American right of centre “big tent” party (as well as the Canadian one) has silenced the voices of the moderate and progressive right while the American “big tent” left of centre party has marginalized the social democratic left.

The Fifth Column contends that a Proportional Representation system does a better job of electing legislatures that represent the views of all voters, and governments that represent the views of a majority of voters. For non-legislature single occupant posts (such as president or governors) direct election using ranked ballot voting is preferred.