The evolution of Large Language Models: From Rule-Based systems to ChatGPT

 Large language models have become a topic of immense interest and discussion in recent years. With the advent of advanced artificial intelligence technologies, we now have the ability to create machines that can process and understand human language at a level never before seen. One such technology that has gained widespread attention is ChatGPT, a large language model developed by OpenAI based on the GPT-3.5 architecture.

ChatGPT has been hailed for its ability to generate human-like text and perform a variety of language tasks with high accuracy. But how did we get here? In this foundational blog post, we will explore the history and underlying developments in large language models, tracing their origins from early rule-based systems to the deep learning-powered models of today. We will also examine the key breakthroughs and advancements that have led to the creation of ChatGPT and its predecessors, as well as the potential applications and ethical considerations of these technologies. So join us as we dive into the world of large language models and explore the possibilities they hold for the future of communication and AI.

The development of large language models has come a long way from early rule-based systems that relied on explicit rules to generate text or respond to queries. The advent of statistical language modeling techniques in the 1980s and 1990s led to the development of the Hidden Markov Model, which was used to model the probability of a sequence of words occurring in a given context.

In the 2000s, neural network-based language models began to emerge, which were able to learn more complex patterns and relationships between words than traditional statistical models. The introduction of the recurrent neural network (RNN) was a significant breakthrough in this area, as it was able to model sequential data such as language with great success. The long short-term memory (LSTM) and gated recurrent unit (GRU) architectures further improved the ability of RNNs to model long-term dependencies in language.

The advent of deep learning in the 2010s led to the development of even more powerful language models, including the Transformer architecture, which was introduced in 2017. The Transformer was able to learn even more complex relationships between words and achieve state-of-the-art results on a wide range of natural language processing tasks.

Today, large language models like ChatGPT have gained significant attention for their ability to generate human-like text and perform a variety of language tasks with high accuracy. These models are typically trained on massive datasets of text and use self-supervised learning techniques to learn representations of language that can be fine-tuned for specific tasks.

While the potential applications of large language models are vast, there are also ethical considerations and limitations to consider. For example, the potential for bias in training data and the impact of these models on the job market are just a few of the challenges that must be addressed as we continue to develop these technologies.

©EverythingElse238

Eighteenth and Nineteenth Century art movements: Rococo, Neoclassicism, and Romanticism

A graphical overview of the three most important art movements of the 18th and 19th centuries.

 

Neolithic-, Chalcolithic-, Bronze-, and Iron-Age civilizations and Art

 

Art of prehistoric and Predynastic Egypt

I have earlier provided an overview of the origins, timeline, and examples of prehistoric art but, due to the importance of Egyptian art in the development of Greek and Roman art, a detour into Egyptian art of this period is warranted.

According to metmuseum.org, "In Egypt, millenia before the advent of powerful dynasties and wealth-laden tombs, early settlements are known from modest scatters of stone tools and animal bones at such sites as Wadi Kubbaniya."

"The Predynastic Period of Ancient Egypt corresponds to the late Neolithic and covers the cultural and social changes which occurred between the late Paleothic period and the early Pharonic era" (thoughtco.com). Prior to the Predynastic, however, there were Paleolithic and Neolithic groups stretching back thousands of years. Both the prehistoric and Predynastic periods are divided into cultural eras, each named after the place where a certain type of Egyptian settlement was first discovered. The chart below shows the various cultures that are included in the two periods.

"During the Predynastic Period, Egyptians developed a written language ... and institutionalized religion. They developed  a settled agricultural civilization along the fertile dark soils of the Nile (which involved the revolutionary use of the plough) during a period in which Northern Africa was becoming arider and the edges of the Western (and Saharan) desert spread" (thoughtco.com). Also during this period, "ceramic figurines, mace heads, and other artifacts such as slate palettes used for grinding pigments, begin to appear, as does imagery that will become iconic during the Pharonic era ... " (Calvert, et al., Smarthistory guide to Ancient Egyptian art, Smarthistory, 2019.)

Examples of art of this period are depicted in the chart below.

@Everythingelse238

Piet Mondrian: The factors influencing his move from Amsterdam to Paris

The paintings Piet Mondrian executed upon his return to Amsterdam from Brabant in 1905 differed from contemporary Netherlands art in that it had "a pronounced rythmic framework and a compositional structure rather than toward the traditional picturesque values of light and shade," but between 1905 and 1911, he would be subjected to a number of experiences that would further influence his painterly direction and eventually lead to his relocation to Paris to pursue what he saw as the future of art.

The first of these experiences was a 1905 Vincent van Gogh exhibition organized by Jo, the sister-in-law of the deceased artist. Jo had been married to Theo for only two years when he died "leaving Jo with a baby and an apartment filled with Vincent's artwork." Theo's goal had been to elevate the visibility of his brother's artwork; with his passing, Jo took on that task. One of her most ambitious efforts was the largest ever retrospective of Vincent's work held at Amsterdam's Stedelijk Museum in 1905. The exhibition, which featured 480 pieces of van Gogh's work, made an enormous impression on Mondrian.

The second experience was the Quadrennial Exhibition held in Amsterdam in 1907 and featuring the Post-Impressionist works of Kees van Dongen, Otto van Rees, and Jan Sluijters. "The forceful expression and use of color inherent in Post-Impressionism are apparent in such works of the period as Red Cloud and Woods near Oele ..."

Red Cloud, 1907
Piet Mondrian

Woods near Oele, 1908
Piet Mondrian

Mondrian adopted several traits of Post-Impressionism and Pointilism as shown in two of his works from the period: Windmill in Sunlight and The Red Tree. For example, he reduced his palette to the basic hues, with Windmill in Sunlight created mainly in yellow, red, and blue.

Mill in Sunlight: The Winkel Mill, 1908
Piet Mondrian

The Red Tree, 1908 - 1910
Piet Mondrian

In 1909 Piet organized an exhibition with Cornelis Spoor and Jan Sleijters at Stedelijk Museum where this type of work was shown. The exhibition featured 250 of Mondrian's works and juxtaposed his early efforts with the later, more modern ones. This show "firmly established the artist as part of the Dutch avant-garde" (prabook.com).

In 1910, his modern works received good reviews at the St Lucas Exhibition in Amsterdam and in 1911 one of his more abstract works was accepted by the Salon des Indépendants.

In 1910 Mondrian co-founded a progressive artists club called Moderne Kunstkring with Jan Sluijters and Conrad Kickert. One of the intentions of the club was the curation of an exhibition of Dutch and foreign artists in Amsterdam. In preparation for the first such exhibition, Mondrian traveled to Paris in 1911 to see the latest artistic trends. 

The group's first exhibition was held at the Stedelijk Museum in 1911 and was the first ever museum presentation of Cubism. It featured artists such as Georges Braque, Raoul Dufy, Le Fauconnais, Pablo Picasso, and Maurice de Vlaminck. Mondrian was deeply impressed with the Cubist works and decided to move to Paris, the home of Cubism.

@EverythingElse23e

Piet Mondrian: The early years of a seminal modern art figure

Today Piet Mondrian "is considered a seminal figure within the development of abstraction, and his oeuvre exerted significant influence on subsequent artists and movements, including the Bauhaus and later Minimilism. His works are included in numerous museum collections worldwide ..." (Sothebys.com). Before arriving at the point of his renown, however, Mondrian "went through many stylistic phases, as his paintings evolved from landscapes and still lifes that looked backwards at time-honored Dutch traditions, to the scintillating geometric canvasses for which he remains best known today." I will explore Mondrian's life and works in a five-part series inclusive of this post.

Tableau I, 1921
Piet Mondrian

Before I begin my exploration of Mondrian's life and works, let us take a look at the Dutch art world that existed at his birth. First, there was a hierarchy in terms of painterly works, with history paintings (included mythology, allegory, and religion; was generally done on a larger scale; and also included recent history) at the top of the pecking order followed by portraiture, genre painting (figures engaged in ordinary activities), still life, and landscape, in that order. There were some aspects of this art hierarchy that were pronounced/unique to Holland: (i) In the still-life arena, animal paintings were popular and (ii) landscapes tended more towards seascapes.

The requirements of the Calvinist theology led to the formation of a new painting regime focused on church interiors. Calvinism held that imagery was idolatrous and, as such, no paintings were allowed in churches. Instead, the focus in the interior of the church was on geometric harmony and balance and artists of the day sought to capture those structures and effects on canvas.

Mondrian was born on March 7th, 1872, in Amersfoort -- a city and municipality in the province of Utrecht -- the second of five children born to Pieter Cornelius Mondriaan and Johnanna Christina Mondriaan. His father was a devout Calvinist who spent his entire career teaching drawing in Calvinist schools. When Piet was 8 years old, his father was promoted to Headmaster of a Calvinist primary school, prompting a family move to Winterswijk. Piet lived in that town until he was 20 years old.

Both his father and uncle gave him guidance and instruction when he began studying drawing at age 14. His uncle -- Fritz Mondriaan -- belonged to the Hague School of landscape painting and had been taught by William Marz, a famed member of that selfsame school.

Piet was bent on becoming a painter but sought to mitigate the concerns of his family by gaining a degree in education first and by 1892 he was qualified to teach drawing in secondary schools. In furtherance of his painting goals he began taking lessons in 1892 in a small town close to his residence. He subsequently moved to Amsterdam to register at the Rijksacademie where he excelled. While in Amsterdam, Piet earned his keep by selling copies of old masterpieces.

He became a member of the Kunstliefde art society in Utrecht and had his first paintings exhibited in 1893. He joined two local artist societies in Amsterdam in 1894 and had a second exhibition of his works in 1897.

Piet Mondriaan

Up to the turn of the century, Piet's paintings followed the trajectory of contemporary Netherlands art. He painted landscapes and still lifes of the areas surrounding Amsterdam.

Wood with Beech Trees, 1899
Piet Mondrian

Mill of Heeswijk Sun, 1904
Piet Mondrian

In 1903 he visited a friend in Brabant and stayed over until the following year. The "calm beauty and cleanliness of the landscape" impacted him, as evidenced by the nature of his work post his return to Amsterdam in 1905. The landscapes he began painting had a "pronounced rythmic framework and a compositional structure rather than toward the traditional picturesque values of light and shade."

Willow Grove: Impression of Light and Shadow, c. 1905
Piet Mondrian

Piet had his first one-man exhibition in 1909, the same year that he became a member of the Theosophic Society. The Society was a spiritual organization centered around the teachings of Buddhism with "prevalent influence in Europe at the start of the 20th century." His involvement "led to a level of spirituality in his paintings which he believed helped his ... audiences reach a holy type of realm when viewing his works." "His involvement within Theosophy went on to influence Mondrian's goal of portraying absolute harmony in his works."

@EverythingElse238

Art + History: The Third of May 1808 (1814) by Francisco Goya

Paul Glenshaw is one of the premier storytellers at the Smithsonian Institution; his Art + History series serving as proof of that assertion. I have previously reported on his skills in recounting his description of The Gross Clinic and I evidenced them again in his recent presentation on The Third of May 1808, Francisco Goya's iconic 1814 painting. 

The Third of May, 1814
Francisco Goya

The subject work was created in Madrid in 1814, went into storage in the Prado in 1834, and had its first known public display therein in 1872. The painting depicts the execution of individuals who participated in a May 2nd (1808) uprising against Napoleon. In this post I provide Glenshaw's context (supplemented with some secondary research) for the painting and its components.

Francisco Goya

Francisco Goya was born on March 30, 1746 in Fuendetodas (his mother's village) to a lower middle class family. His father being a gilder, he was born into a world of creativity. He spent the majority of his childhood in Zaragossa where he began his artistic studies -- at the age of 14 -- under the painter José Lugán. Those studies were completed after 4 years after which Francisco went to Madrid to study under Anton Raphael Mengs, a German who was Court Painter to the Spanish Royal Family.

According to franciscogoya.com, Goya clashed with his Master and his examination was unsatisfactory. He submitted entries for the Royal Academy of Fine Arts in 1763 and 1766 but failed on both occasions.

Portrait of Goya, Vicente Lopéz Portana, c.1826

Francisco studied in Rome and travelled all over Europe. One of his early works from this period was Sacrifice to Pan (1771)

Sacrifice to Pan, 1771

Francisco Goya

His painting The Victorious Hannibal came in second in a painting competition organized by the City of Parma.

Hannibal the Conqueror, Viewing Italy for the first time from the Alps, 1770 - 1771
Francisco Goya

He returned to Zaragoza in 1771 and painted parts of the cupola of the Basilica of the Pillar (including Adoration of the Name of God).

Adoration of the Name of God, 1772

Francisco Goya 

In Zaragoza Goya studied with Francisco Bayeu and eventually married the latter's sister Josefa with whom he had eight children. This relationship (Bayeu was a member of the Royal Academy of Fine Art) helped Goya to secure work as a painter of designs to be woven by the Royal Tapestry Factory. His work on the patterns brought him to the attention of the Royal Court (franciscogoya.com):

• Appointment as a member of the Royal Academy of Fine Art
• In 1783, the Count of Floridabianca commissioned Goya to paint his portrait
• He spent two summers painting portraits of the Crown Prince Don Luis and his family
• He painted portraits of the Duke and Duchess of Osuna, the King, and other notables during the 1780s
• In 1786 he was given a salaried position as painter to Charles III
• In 1789 he was made Court Painter to Charles IV
• In 1799 he was appointed First Court Painter
• In addition to the Royal Family, Goya received commissions from many members of the Spanish nobility.

In 1792 Goya left Madrid for Cadiz where he was struck down by a mysterious illness which manifested as dizziness, hallucinations, and, eventually, a loss of hearing.

The Political Environment

The French Revolution was launched around this time with Napoleon eventually taking the reins of the country. As it relates to geopolitics, England was allied with Portugal while Spain was weak. This state of affairs concerned Napoleon as he feared that Spain would fall to England and France.

In 1808, Fernando, son of King Charles IV, staged a coup and forced his father to abdicate. Napoleon had Charles and Fernando meet with him to discuss the political crisis and forced them to abdicate in favor of Napoleon's brother Joseph. Both Fernando and Charles were sent into exile. Napoleon then sent in French troops to solidify the takeover.

On May 2nd, the local population exploded in fury at the state of affairs and proceeded to attack the French troops. The Spanish troops handed out weapons to the upstarts so that they could battle the French on a more equal footing. 

On the 3rd of May additional French troops were deployed to bring the uprising under control. Recalcitrants were rounded up and shot.

Goya's Rendition of Events

Years later Goya was given a commission to commemorate the event. He began with the events of the 2nd of May, showing the uprising in the center of the city with Spanish citizens battling the Mamelukes and French officers. The scene is chaotic, with dead bodies on the ground juxtaposed against the action of battle and the solidity of the background architecture.

The Second of May 1808, 1814
Francisco Goya

The painting Third of May shows the events of the following day. 

Third of May 1808, 1814
Francisco Goya

There is a hill and a building in the background and people are coming into the foreground from that space. The foreground is dominated by a firing squad on the right side of the painting, a lantern in the center, and targets/victims on the left.

The detail below focuses on the members of the firing squad. In this detail we see humans as killing machines: no faces; similar clothing; and almost merged guns. The knapsacks are different so they are individuals. No feelings are exhibited here. The ground around their feet are illuminated by the lantern  but only two partial shadows are cast.

Detail of Francisco Goya's Third of May 1808

In the detail below we see the square lantern lighting up the victims, showing what has already been done; and what is to come. The ground in front of the victims is splattered with the blood of the dead and wounded while those still standing are in various stages of acceptance of their fates. The central figure is clothed in white and gold and stands, eyes bulging and hands upraised. The light from the lantern emphasizes this individual.

Detail of Francisco Goya's Third of May 1808

Goya never exhibited any political inclinations during his life but apparently thought that those May events should be memorialized. According to artincontext.org, it was Goya who presented the idea of the paintings to the Spanish government and they commissioned the works.

According to artincontext.org, this painting "... is widely acclaimed by many art sources as being one of the 'first' modern art paintings" in that "the subject matter and artistic technique depart from what was expected at the time from history paintings ..." and influenced a number of artists ( Edouard Manet, The Execution of Emperor Maximilian (1868 to 1869); Pablo Picasso, Guernica (1987); Pablo Picasso, Massacre in Korea (1951); and Robert Ballagh, The Third of May after Goya (1970)) to produce "paintings related to execution from war."

©EverythingElse238