We know that the worst bear market in United States history occurred between 1929 and 1932 when the S&P Composite fell 86%, returning the stock market back to levels it hadn’t seen since the 1800s.  But what is the longest bear market and how long did it last? To answer this question, you have to understand GFD’s definition of a bear market as a 20% decline in the stock market and a bull market as a 50% increase in the stock market.  If the stock market falls by 30%, rises by 40%, then falls by another 30%, that would be treated as a single bear market.  So you could rephrase the question as what is the longest period of time in which the stock market failed to rise by at least 50%? Now that Global Financial Data has calculated its US-100 index that stretches from 1792 until 2019, we can answer that question.  The answer is 51 years going from the establishment of the stock market in Philadelphia in 1792 when the Bank of the United States (BUS) came into existence until the 1843 market bottom after the Panic of 1837.  During that period of time, the stock market lost over two-thirds of its value.  The only bear market with a larger decline was the bear market of the Great Depression mentioned above.  

Banks, Insurance Companies and the 51-Year Bear Market

You have to remember that during the first half of the 1800s, most of the stocks that traded in the United States were bank and insurance companies and most of the return to stocks came in the form of dividends.  Between 1792 and 1843, stocks on average lost 1% per annum which compounded to a 66% loss over the 51-year period as is illustrated in Figure 1.  However, if the dividends stocks paid had been reinvested, the return would have been positive with $100 growing to $690, not declining to $33. This provided a gross return of 3.86% per annum, not a loss of 1% per annum. It should also be remembered that two stocks, the First and Second Banks of the United States were by far the largest companies in the United States during those 50 years.  The Bank of the United States represented over 80% of the stock market capitalization in 1792 and at least 20% of the total stock market capitalization until 1843.  The First Bank of the United States rose to $600 after shares were issued, then slowly declined in value to its $400 par by 1815. Similarly, the Second Bank of the United States jumped in price to $150 when shares were first issued, then traded between $100 and $125 until President Andrew Jackson failed to renew its charter in 1836. The bank went private, but failed and the stock collapsed in price along with the rest of the stock market after the Panic of 1837. Until the advent of railroads in the 1830s, the stock market was made up almost exclusively of state-chartered bank and insurance companies.  Most of the banks had a single office in Boston, New York, or Philadelphia with little hope of growth.  If the bank or insurance company made a profit, it was paid out in dividends to shareholders, not reinvested for expansion. During a panic, as in 1819 or 1837, some banks went bankrupt, driving the index down. Between 1792 and 1843, there were two Panics that hit the United States in 1819 and in 1837, both of which drove the market down.  After the United States failed to recharter the First BUS, the United States encouraged banks to lend money during the War of 1812.  After the Second Bank of the United States was chartered in 1816, it began lending money freely, but in the summer of 1818, the Bank initiated a sharp credit contraction.  The BUS began rejecting state-chartered banknotes in August 1818 and in October 1818, the US Treasury demanded a transfer of $2 million from the BUS to redeem bonds on the Louisiana Purchase.  The price of cotton fell 25% in one day in January 1819 and the Panic was on. Between June 1818 and June 1819, the US-100 index fell by 25%. Stocks recovered modestly in the 1820s and 1830s, moving back to the level they had been at in 1818. But the economy grew too quickly. States such as Mississippi issued bonds in London, and cotton exports began rising, bringing silver into the United States promoting economic growth. Meanwhile, the United States expanded westward into new territories and states leading to land speculation and higher prices.  The Bank of England decided to raise interest rates in 1836 to increase its monetary reserves, and New York banks followed suite in early 1837.  The price of cotton fell by 25% in February and March of 1837.  The Specie Circular was issued in 1836 requiring all land purchases to be paid for in specie, not in banknotes, putting a brake on rising real estate prices.  The Deposit and Distribution Act of 1836 put federal funds in western banks, reducing deposits in money center banks in New York and Philadelphia.   When Martin Van Buren became President in March 1837, he refused to provide emergency relief or increase federal spending to slow the downturn leading to a general decline in the American economy that persisted until 1843. In the 1830s, Americans began building the railroads that would eventually crisscross the nation.  The Baltimore and Ohio was founded in 1828 and other railroads began popping in New England and along the Atlantic seaboard. The railroads pulled the American economy out of its recession, and the United States enjoyed growth until the Panic of 1857 led to the next recession.  

The Longest and Mildest Bear Market?

The first bear market in the United States was as much a factor of the nature of the stock market as anything else.  Two Panics in 1819 and 1837 drove the stock market down, but until the railroads came along, there were few growth opportunities that could push the market up into a bull market.   Small banks and insurance companies that had little prospect for growth dominated the American economy.  Profitable banks simply paid out dividends to shareholders while unprofitable banks collapsed.  The result was an overall decline in the price of banks of about 1% per annum, but with banks paying dividends that averaged 5%, investors still received a 4% total return on average. While the recovery from the Panics of 1819 and 1837 were mild, the recovery from the Panics of 1857, 1873 and 1893 were sharp as Figure 1 illustrates. After 1840, manufacturing companies grew up around Boston and railroads were built to connect American cities to each other.  At that point, the regular pattern of alternating bull and bear markets began and continues to this day.  By our count, there were four bear markets in the 1800s, twelve in the 1900s and two, so far in the 2000s.  How many more bear markets will occur before the century ends we do not know, but we can guarantee you that no 50-year bear market is likely to ever occur in the United States again.

Global Financial Data has calculated a 100-share index that is more comprehensive than existing long-term indices of the United States stock market.  GFD has collected historical data on U.S. shares from 1791 until 2017 and is using this database to calculate a more representative index of U.S. shares than is currently available. The GFD-100 Index is superior to existing indices for three reasons.  First, the index is capitalization weighted from its beginning in 1791 until 2019.  Second, the selection of stocks is based upon all shares that were traded on U.S. exchanges and over-the-counter. The selection of shares is not limited to the New York Stock Exchange (NYSE), but includes shares from regional exchanges and over-the-counter markets. Third, the index includes finance stocks throughout its history.  Finance shares were excluded from the S&P 500 Composite until July 1, 1976 when the composition of the S&P 500 was changed from 425 Industrials, 60 Utilities and 15 Rail shares (adopted in March 1957) to 400 Industrials, 40 Utilities, 20 Transportation and 40 Finance stocks. Few people realize that bank and insurance stocks were excluded from any major United States stock index until 1976.  Although bank and finance stocks mostly traded over-the-counter before the 1970s, finance stocks represented between 10% and 20% of the overall capitalization of the stock market before 1976. Bank and insurance companies were excluded because consistent price quotes were not available and finance stocks were not as liquid as the shares traded on exchanges. The exclusion of over-the-counter shares was not limited to banks and insurance companies. Standard Oil and its subsidiaries also traded over-the-counter and were excluded from stock market indices until they moved onto the New York Stock Exchange. Nevertheless, people did invest in bank and insurance stocks, and in the Standard Oil companies, and their exclusion creates biases in the historical calculations of stock market indices. Existing calculations of long-term stock market returns in the United States are based upon four primary sources: Smith and Cole (1803-1862), Macaulay (1857-1871) Cowles (1871-1928) and Standard and Poor’s (1928-2017). Unfortunately, these indices have major flaws in them that create biases that researchers have tolerated until now because no one has ever collected historical data on U.S. share prices, corporate actions (dividends and splits) and shares outstanding so accurate price and return indices could be calculated. The current S&P Composite includes data from the Cowles Indices from 1871 until 1927, the 90-share daily S&P Index from January 1928 until February 1957, the 500-share composite that excludes finance stocks until July 1976 and the all-sector 500-share index since July 1, 1976. Global Financial Data has collected data on all major United States exchanges going back to 1791 as well as data on over-the-counter shares since 1865.  The data includes not only share price data, but information on corporate actions (dividends and splits) as well as shares outstanding.  These data sources enable us to calculate cap-weighted price and return indices for the United States that accurately reflect what an investment in the 100 largest stocks each year would have produced for investors. GFD set up criteria for determining which stocks are included or excluded from its indices.  GFD has followed these rules for inclusion: 1) there had to be at least 9 observations per year, 2) Dividend data have to be available for each stock in order that both price and return indices can be calculated, and 3)  There had to be share outstanding information available so the stock could be included in a capitalization-weighted index. The index includes all shares that met these criteria from 1791 to 1824, the top 50 shares by capitalization are used from 1825 to 1850 and the top 100 shares by capitalization from 1851 to 2017. To choose which companies to include, shares were weighted by capitalization during the first month of each year and included if they were among the 100 largest shares in the United States. It was assumed that the stocks were held for the rest of the year, and in January of the next year, the same selection methodology was used to choose which shares to hold for the coming year.  Each year, the list of the 100 largest companies was recalculated and a new list of stocks was introduced.  For continuity purposes, if a stock missed a year, i.e. the stock was in the top 100 in 1914 and 1916, but not in 1915, the stock was included in the index in 1915 even though this put over 100 stocks in the index. A decade-by-decade comparison of the return to stocks in the GFD-100, bonds in GFD’s U.S. Bond Index, bills in GFD’s US Bill Index and the equity-risk premium is provided below.

Decade	Stock Price	Stock Return	Dividends	Bonds	Cash	Equity Premium
1792-1799	-3.7	2.67	6.61	-0.15	5.75	-2.91
1800-1809	1.35	8.01	6.57	6.03	4.96	2.91
1810-1819	-4.35	1.59	6.21	7.42	4.99	-3.24
1820-1829	0.53	5.82	5.26	5.01	3.66	2.08
1830-1839	-0.95	5	6.01	0.44	4.51	0.47
1840-1849	-0.8	6.12	6.98	6.97	4.94	1.12
1850-1859	-0.58	6.65	7.27	4.67	5.01	1.56
1860-1869	4.62	12.45	7.48	9.28	4.97	7.13
1870-1879	2.04	8.97	6.79	5.52	3.82	4.96
1880-1889	1.19	6.27	5.02	4.16	3.01	7.98
1890-1899	4.05	9.45	5.19	4.57	2.13	2.81
1900-1909	6.25	11.23	4.69	0.76	3.01	7.98
1910-1919	-0.69	5.5	6.23	2.22	2.62	2.81
1920-1929	8.72	14.44	5.26	5.53	3.5	10.57
1930-1939	-3.48	0.5	4.12	6.32	0.27	0.23
1940-1949	3.43	8.21	4.62	2.25	0.58	7.59
1950-1959	13.67	18.3	4.07	0.70	2.16	15.80
1960-1969	3.82	6.83	2.90	1.28	4.4	2.33
1970-1979	2.14	6.12	3.90	4.09	6.71	-0.55
1980-1989	12.94	18.14	4.60	15.54	7.96	9.43
1990-1999	19.78	22.65	2.40	7.20	4.52	17.35
2000-2009	-1.66	0.16	1.85	5.42	2.25	-2.04
2010-2017	7.67	10.06	2.22	3.67	0.21	9.83
Average	3.30	8.48	5.06	4.73	3.74	4.60

  Although the overall returns between 1870 and 2017 do not differ significantly between the GFD-100 and Cowles/S&P Composite, there are several advantages in using the GFD-100 as the benchmark for long-term historical data for the United States stock market rather than the Cowles/S&P Composite. 1.     The GFD-100 uses shares that traded on all United States exchanges and over-the-counter from 1791 until 2017. The Cowles/S&P Composite is limited to the New York Stock Exchange before 1972.  Finance companies that traded OTC and companies such as Standard Oil which traded OTC for several decades before moving to the NYSE are included in the GFD-100, but excluded from Cowles/S&P. 2.    The GFD-100 uses shares from all sectors, including finance, from 1791 until 2019.  The Cowles/S&P Composite only includes finance shares beginning in 1976 and ignores the finance sector before 1976. 3.    The GFD-100 includes accurate data on dividends from 1791 to 2019.  The Cowles/S&P Composite only calculated dividends from 1871 until 2017.  There is no inclusion of dividends before 1871 in the Cowles/S&P Composite because no data on dividends were collected. Consequently, existing indices are missing 80 years of dividend data. 4.    The GFD-100 is capitalization weighted from 1791 until 2019.  The Cowles/S&P is cap-weighted from 1871 until 2019 and includes no capitalization weighting before 1871. 5.    The Smith and Cole bank indices that cover the period from 1802 until 1845 suffer from survivorship bias.  Banks were chosen for the indices based upon their longevity, not on their size or liquidity. The GFD-100 components are chosen based upon their market capitalization.  The largest companies are chosen during each January and are “held” in the portfolio for the rest of the year when a new portfolio is organized for the coming year. 6.    The Smith and Cole indices are based upon a very limited population of six to eighteen companies per year from 1802 until 1862.  The GFD-100 includes 50 companies from 1825 until 1850 and 100 companies from 1851 using a broader population of shares. 7.    The GFD-100 uses a consistent methodology from 1791 until 2019.  The Cole and Smith/Macaulay/Cowles/S&P Index use different methodologies.  The data that are used to put together the composite are collected from four different sources and chain-linked together in an uncertain pattern.  During the periods of time when different indices exist, choosing different indices generates different rates of return. Overall, the GFD-100 provides a superior benchmark stock index.  Because of its greater accuracy, we would encourage financial historians to use the GFD-100 for their analysis of long-term trends in the stock market in the United States.

The Wilshire 5000 is the broadest index in the United States covering every stock that is listed on the New York Stock Exchange, NASDAQ and the NYSE AMEX.  When the Index was launched back in 1974, the index included about 5000 stocks, hence the name.  The Wilshire 5000 was never limited to 5000 stocks, but includes every stock listed on the three exchanges regardless of the total number of companies. Wilshire also created the Wilshire 4500, which includes all of the stocks that are listed on the three U.S. exchanges excluding the stocks in the S&P 500 as well as the Wilshire 2500, which includes the 2500 largest companies.   Wilshire’s Large Cap Index includes the top 750 stocks by capitalization while the Small Cap Index includes the 1750 stocks after the first 750. The Micro-Cap Index includes all stocks included in the Wilshire 5000 that are not included in the Wilshire 2500.  

Growth in the Wilshire 5000

As the stock market boomed in the 1990s, the Wilshire 5000 grew in size and reached its maximum membership on July 31, 1998 when 7,562 companies were included in the index.  Thence, membership has shrunk, declining back to 5000 on December 29, 2005. Since then, the Wilshire 5000 has included less than 5000 companies. The Wilshire 5000 Full Cap Index calculates the value of all of the shares listed in the United States in billions of dollars.  The index was at 28,552.5 on February 15, 2019 meaning that the value of all the common stocks listed in the United States was $28,552.5 billion. The index reached $30 trillion in September 2018 before the current sell off.  The index has data going back to December 1970 when the total capitalization of the US Stock market was $830 billion, though this sank to a low of $550 billion in September 1974. GFD has used its own calculations of all the stocks listed in the United States to extend the Wilshire 5000 back to 1792. So to answer the question, how many stocks are in the Wilshire 5000? On June 30, 2018, there were 3,486 companies.  This means that the number of companies that are included in the Wilshire 5000 has shrunk by 54% Since July 1998.  Figure 1 illustrates the performance of the Wilshire 5000 since 1970.  The index showed strong growth between 1975 and 1999, but slower growth since then.

Large Caps vs. Small Caps

It is also instructive to compare the performance of the S&P 500 Total Return Index with the Wilshire 5000 Total Market Index, since both are total return indices.  This comparison provides us with the best comparison of the performance of large cap and small cap stocks that is available. If the S&P 500 stocks (large caps) outperform the rest of the market, then the relative index rises, if the rest of the stocks (small caps) outperform, it falls.  As Figure 2. Illustrates, the S&P 500 peaked relative to the rest of the market in 1975, declined until 1983, then began a stead rise relative to the rest of the market until 2001.  Between 2001 and 2014, small caps outperformed the S&P 500, but during the past 5 years, the S&P 500 has been gaining strength relative to the rest of the market.

The Wilshire/Russell 3000?

The number of stocks that are listed in the United States continues to decline. Because of the regulatory cost of listing on public exchanges, and the willingness of venture capitalists to invest in private companies, fewer and fewer companies are going public, and the number of publicly-traded companies is declining.  The Russell 3000 may soon face the fate of the Wilshire 5000, being forced to include fewer companies than its name suggests.  At some point in the near future, there may be fewer than 3000 companies listed in the United States and at that point, the Wilshire 5000 and the Russell 3000 will include the same number of companies.  When that happens, we will let you know.

Global Financial Data has put together a unique collection of alternative data indices that chart the performance of stocks in dozens of different countries around the world.  The first company for which Global Financial Data has data, the Dutch East India Co., was founded so the Dutch could develop resources in Indonesia and the rest of Asia.  This company was followed by the Dutch West India Co., the English East India Company, the French East India Co., the Hudson Bay Co. and dozens of others.  Throughout the 1600s and 1700s, developing the resources of emerging markets was the main reason many of these companies existed. During the 1800s, however, European countries established corporations that had the right to build railroads, establish banks, mine, and purchase and develop land.  Each country established companies to develop resources in their colonies, Britain in India and Australia, the Netherlands in Indonesia (Netherlands Indies), France in French Indochina and French West Africa, Belgium in the Congo, and so forth. Today, if you want to invest in Emerging Markets, you can either buy an ETF that specializes in a particular country or region, or you can purchase ADRs that provide you ownership in that company. In the 1800s, there were no ETFs that allowed you to invest in different colonies or regions, but you could buy shares in companies that invested in emerging markets.  These shares were liquid and readily available on the London Stock Exchange. There were numerous bubbles that attracted British and other investors to emerging markets before World War I.  The first occurred in 1825 when Latin American companies caught the eyes of investors.  French railroads raised money in London in the 1840s, Australian mining stocks exploded in 1852, American railroads attracted investors in the 1870s and 1880s and in the 1890s, South African gold mining companies traded on the London, Paris and Berlin stock exchanges.  The Suez Canal was one of the largest companies that listed in Paris and the Panama Railroad was a prime subject of speculation in the 1850s as gold diggers made their way from the east coast of America to California. The London Stock Exchange is so rich in its history that you might feel like an archaeological explorer who has discovered Tutankhamun’s tomb once you start investigating the data.  In most of the countries, shares listed in London before stock exchanges established in those countries.  Since the gold standard fixed exchange rates between European countries after 1870, shares of American railroads or South African mining companies traded simultaneously in London, Paris, Amsterdam, Berlin, Brussels and Vienna.  Until 1914, buying and selling government bonds or the bonds of American railroads was one of the principal ways that money moved between countries to arbitrage exchange rates between European countries. The coverage varies tremendously between different countries.  Some countries, such as Costa Rica or Ethiopia, may only have one company that traded in London, but other countries, such as Australia, Malaysia, South Africa or the United States might have over 100 companies that traded in London between 1825 and 1985.  Global Financial Data has collected the price of shares, the number of shares outstanding and corporate actions so market cap-weighted price and total return indices can be calculated for each country. The data allows us to calculate indices not only by country, but by region.  We can compare the performance of developed and emerging markets as well as by sector. Were emerging markets good investments in the 1800s or poor investments?  Did the returns to emerging markets exceed returns to British or American companies?  Did people in emerging markets benefit from the railroads, banks, mining and other companies that were developed, or as Marx or Gandhi would have it, did these companies channel all their profits back to European investors and leave nothing for the colonies? Over 2300 companies that listed in the United Kingdom and the United States are included in GFD’s London Stock Exchange indices.  Certainly, no one who wants to understand the performance of the stock market in the past or of emerging markets in general should ignore this data.  The results can be compared directly with the returns to the United States and United Kingdom using the GFD US-100 and GFD UK-100 indices. The data from London and New York are daily in their periodicity so the exact timing of bull and bear markets can be studied in detail. Once you have analyzed the indices that have been calculated for each country or sector, you can analyze the performance of the underlying companies that make up the indices to discover the sources of changes in the data.  The GFD Indices provide a rich resource of alternative data that is available from no other source and can be a rich resource for finding sources of alpha for investors. To help potential users understand the richness of this data set, Table 1 provides information on the indices that have been calculated for each country.  The table details when the index for each country begins and when it ends.  It tells the number of companies that are included in the indices as well as the maximum market cap of the companies that were listed in London. If you currently do not subscribe to the GFD Indices, you are missing out on a rich opportunity to analyze the past and possibly obtain a better understanding of the performance of emerging markets in the future. Table 1.  GFD Indices of Companies Listed in London and New York, 1825 to 1985

Country	Begins	Ends	Companies	Max Market Cap	Year
Argentina	1865	1985	69	749.0	1929
Australia	1825	1985	201	12,550.0	1981
Austria	1856	1932	6	27.0	1906
Belgium	1845	1985	8	21.0	1875
Bolivia	1825	1985	8	100.0	1929
Brazil	1825	1984	60	2,563.0	1984
Canada	1825	1985	100	19,278.0	1976
Chile	1852	1969	46	815.0	1929
China	1882	1930	7	38.0	1925
Colombia	1825	1962	21	33.0	1920
Costa Rica	1886	1932	1	1.3	1890
Cuba	1838	1961	23	135.0	1926
Denmark	1853	1984	4	40.0	1937
Ecuador	1924	1975	1	2.3	1944
Egypt	1856	1969	23	918.0	1929
El Salvador	1887	1985	4	5.4	1978
Ethiopia	1908	1924	1	0.2	1920
France	1801	1985	53	881.0	1890
Germany	1835	1985	17	7,647.0	1985
Ghana	1869	1985	21	175.0	1936
Greece	1839	1930	3	15.0	1911
Guatemala	1923	1963	2	10.0	1956
Guyana	1845	1921	1	0.1	1895
Hong Kong	1865	1985	6	272.0	1969
India	1792	1985	156	530.0	1865
Indonesia	1891	1981	40	157.0	1980
Ireland	1792	1985	87	402.0	1877
Italy	1848	1985	19	78.0	1927
Jamaica	1907	1985	6	3.0	1967
Japan	1907	1985	44	68.0	1913
Kenya	1907	1970	7	31.0	1966
Malaysia	1889	1985	212	3,373.0	1981
Mauritius	1854	1915	5	2.0	1878
Mexico	1824	1985	61	2,112.0	1989
Mozambique	1895	1975	2	3.5	1928
Myanmar	1890	1977	6	95.0	1937
Netherlands	1845	1985	25	19,106.0	1985
New Zealand	1862	1980	32	219.0	1974
Nicaragua	1863	1891	2	3.3	1864
Nigeria	1887	1976	34	53.0	1965
Paraguay	1889	1965	3	10.0	1965
Peru	1825	1975	13	317.0	1968
Philippines	1889	1985	10	954.0	1980
Portugal	1855	1981	8	13.0	1922
Romania	1870	1940	3	10.0	1914
Russia	1865	1932	22	80.0	1917
Singapore	1895	1977	5	6.0	1969
South Africa	1833	1985	379	54,785.0	1980
Spain	1845	1985	21	90.0	1913
Sri Lanka	1842	1984	57	151.0	1927
Sweden	1853	1985	21	334.0	1929
Switzerland	1872	1930	1	0.5	1928
Thailand	1919	1970	2	11.5	1964
Trinidad	1865	1985	8	741.0	1980
Turkey	1856	1930	11	38.0	1929
United States	1821	1985	198	4,584.0	1926
Uruguay	1873	1971	10	30.0	1928
Venezuela	1852	1971	15	305.0	1948
Zimbabwe	1893	1985	43	108.0	1985