What happened this week?
Historical look into the efficacy of tariffs, Deepseek's algorithmic jailbreak rate at 100% and new multichallenge benchmark stress test
Historical look into efficacy of tariffs: Mercantilism, colonial market forces, the Great Depression and Anglo-French trade disputes.
DeepSeek’s algorithmic jailbreak rate at 100%: Cisco & UPenn show DeepSeek failing on all 50 random prompts of the HarmBench Dataset, including on topics like misinformation, cybercrime and illegal activities.
New MultiChallenge Benchmark: Scale AI’s benchmark on multi-turn conversations has DeepSeek-R1 finishing 10th.
Tariffs: Historical Efficacy and Contemporary Insights
Tariffs are taxes levied on imported goods which effectively raise prices of foreign products to disincentivize consumers from purchasing such services in a bid to promote local alternatives. Although this protectionist measure can stimulate local industries, historical analyses provide a deeper insight into long term destabilizing risk factor associated with persistent tariffs.
Optimal Tariff Theory
My interest in Optimal Tariff Theory arose during Scott Bessent’s senate hearing (Min. 34-38), when he responded to Senator Ron Wyden’s mistrust of tariffs. Mr. Bessent stated a 10% tariffs on all commodities leads to a 4% dollar appreciation, thereby offsetting costs to American consumer. Without going into the current political landscape too much, a stronger currency doesn’t always equate to purchasing power as explained below:
Exchange Rate vs. Purchasing Power: A stronger currency should mean that each unit of your currency can buy more of another country's goods and services. However, this is based on nominal exchange rates. Purchasing Power Parity (PPP) aims to adjust exchange rates to reflect the actual purchasing power of currencies in different countries for a basket of goods and services while nominal exchange rates can be influenced by numerous factors including tariffs, including interest rates, inflation expectations, and speculative capital flows.
Imported Goods vs. Domestic Goods: A stronger dollar does make imports cheaper. This is beneficial for consumers who buy imported goods directly. However, many consumer goods in a modern economy, even if branded as domestic, rely on imported components or raw materials. Tariffs increase the cost of these inputs, even with a stronger currency which in-turn can lead to inflationary pressures. Domestic producers, facing less import competition due to tariffs, might also raise their prices. The net effect can be inflationary, even with a slightly stronger currency.
Example: Imagine a 10% tariff on imported steel leads to a 4% dollar appreciation. While the dollar is stronger, the price of imported steel within the US still rises (though by less than 10% due to the currency effect, assuming a full pass-through and ignoring other market dynamics). Industries that use steel as a major input, like auto manufacturing or construction, will face higher costs. Even with a stronger dollar, these higher input costs can get passed on to consumers, potentially negating some or all of the supposed purchasing power gains from currency appreciation.
Optimal Tariff Theory suggests that a country can offset economic burden of commodity prices from domestic consumers to foreign suppliers if the nation is a primary buyer of competing suppliers, or in other words, multiple vendors want to sell to the host country. A problem that arises when the country with market power imposes tariffs, retaliatory tariffs nullify benefits leading to global economic inefficiencies given the current trade scenario. Previously, the United States got 100% of its revenue from tariffs, preceding the existence of income taxes, notably under the William Mckinley (1897-1901) Presidency. However, his Presidency had a drastically different economic context:
Less Globalized Trade: Global trade was far less integrated than it is today and supply chains were shorter with economies being less interdependent.
Manufacturing-Dominated Economies: Economies like the US and Germany were in the midst of rapid industrialization. Manufacturing was the dominant sector, and tariffs were seen as a tool to nurture nascent industries by protecting them from established foreign competitors, primarily from Britain. The focus was on "infant industry" protection.
Limited Service Sector and Intellectual Property: The service sector and trade in intellectual property were much less significant than today. Tariffs primarily targeted manufactured goods and raw materials. This simpler trade structure meant that tariffs had a more direct and predictable impact on specific domestic industries.
Gold Standard and Fixed Exchange Rates: Many major economies, including the US, were on the gold standard or similar systems that aimed for relatively fixed exchange rates. This limited the currency appreciation effect of tariffs compared to today's floating exchange rate system since the US went off the Bretton Woods system in 1971 colloquially known as the Nixon shock during the Vietnam war. Exchange rates were less responsive to trade policy changes then than they are now.
Navigation Acts and Colonial Trade
In the realm of Mercantilism & Colonial Trade, European powers used tariffs to control colonial economies, often enriching the mother country at the colonies' expense. Britain's Navigation Acts (1651-1849) forced colonies to trade only with Britain with the purpose of controlling colonial trade for Britain's benefit. Key acts included:
1651 Act: Goods imported to England and its colonies had to be carried on English ships.
1660 Act: Specified that certain "enumerated" goods (like sugar, tobacco, cotton, wool) from the colonies could only be exported to England or other English colonies.
1663 Staple Act: Goods destined for the colonies from Europe had to pass through England first.
The impact on colonies was net negative:
Economic Stifling: Colonial economies were forced to produce raw materials and agricultural goods for Britain and were discouraged from developing manufacturing sectors that could compete with British industries.
Higher Prices for Goods: Colonists had to buy manufactured goods from Britain, often at inflated prices due to lack of competition. The Staple Act increased the cost of European goods as they had to be shipped via England first.
Lower Prices for Colonial Exports: Colonists were forced to sell their raw materials to Britain, often at prices dictated by British merchants, reducing their profits.
Debt and Resentment: The system fostered economic dependence on Britain and bred resentment due to perceived unfairness and economic exploitation. This resentment was a significant contributing factor to the American Revolution.
The impact on Britain was net positive leading to wealth accumulation from skewed trade, naval dominance of trade routes and manipulated market control.
The Corn Laws and the Repercussions of Protectionism
In Britain during the early 19th century, the Corn Laws (1815–1846) kept grain prices high, benefiting landowners but hurting consumers and industrial workers. These were tariffs and other trade restrictions on imported food and corn imposed because of grain shortages in England as a result of a growing population and blockages imposed Napoleonic Wars. While this protection bolstered the income of landowners, it simultaneously imposed higher living costs on urban workers and industrialists. Economic analyses from the period show that these tariffs contributed to a misallocation of resources, with eventual public and political backlash leading to their repeal in 1846—a move that paved the way for broader free trade policies and spurred industrial growth.
Tariffs & Industrialization
In a historical context, the US-Morill Tariff (1861) helped fund the Civil War while also supporting domestic industries. Southern states, which had generally favored low tariffs, had seceded, leaving Congress dominated by protectionist Northerners and thus in the midst of the Civil War, the Union needed revenue to finance the war effort which were generated from tariffs.
What Happened:
Increased Tariff Rates: The Morrill Tariff significantly raised average tariff rates in the US, reversing a period of tariff reductions in the 1840s and 1850s. Rates on some manufactured goods rose to over 50%.
Revenue Generation: It was successful in generating substantial revenue for the Union war effort.
Industrial Growth (Debated): The extent to which the Morrill Tariff caused US industrial growth is debated by economists. The US was already industrializing rapidly before the Civil War, and other factors like abundant natural resources, technological innovation, and immigration were likely more significant drivers. However, the tariff certainly provided a protective umbrella for Northern industries during a crucial period.
South's Grievances: The Morrill Tariff and the broader issue of tariffs were long-standing grievances for the Southern states, who felt they were being economically exploited by the North’s protectionist policies.
The Great Depression and the Smoot-Hawley Tariff Act
The United States’ Smoot-Hawley Tariff Act of 1930 is perhaps the most cited modern example. This Act raised US tariffs deepening the Great Depression by stifling global trade and prompting retaliatory tariffs worldwide (Canada imposed tariffs on US goods leading to decline in exports). By raising tariffs on over 20,000 imported goods, the Act intended to protect American jobs and industries during the economic downturn. The initial push came from agriculture lobbies to protect farmers and following 1929’s market crash with worsening conditions, the protectionism included manufacturing industries as well. However, export figures dropped by more than 60%, and retaliatory measures by other nations deepened the global economic crisis with global trade declining 25-33%.
Anglo-French Trade Disputes and the Road to Cooperation
In the mid-19th century, high tariffs contributed to prolonged trade disputes between Britain and France. The ensuing economic friction was eventually mitigated by the Cobden-Chevalier Treaty of 1860, which reduced tariff barriers and stimulated trade between the two nations.
Summary of Impacts
Deadweight Loss: Tariffs create "deadweight loss" – a reduction in overall economic welfare. This occurs because tariffs distort market signals, leading to inefficient production and consumption patterns. Resources are misallocated, and potential gains from trade are lost.
Retaliation Risk: Imposing tariffs carries the significant risk of retaliation from trading partners. Retaliatory tariffs can nullify any potential benefits of the initial tariff and lead to a trade war, harming all countries involved.
Short Term versus Long Term Impact: Some industries may experience a temporary boost in production and employment due to reduced import competition however in the long run, tariffs tend to lead to market inefficiencies since industries protected by tariffs may become less competitive and innovative over time. Strategic tariffs are a caveat to nurture important industries until maturation for global competitiveness where specificity and time limits, retaliation risk and dynamic impacts of protectionism to boost local capital allocation all come into play.
DeepSeek R1: Vulnerabilities in Security and Multi-Turn Conversational Benchmarks
As I mentioned last week, DeepSeek is prone to jailbreaking given the lack of focus on safety and no use of supervised fine tuning (SFT), given their RL approach.
Algorithmic Jailbreak Vulnerability: Cisco’s Robust Intelligence and the University of Pennsylvania have exposed critical security flaws, using 50 adversarial prompts from the HarmBench dataset, designed to probe for harmful outputs related to misinformation, cybercrime, and illegal activities. DeepSeek R1 demonstrated a 100% algorithmic jailbreak rate. Confirmed independently by security firm Adversa AI, signifies that DeepSeek R1 failed to block any of the harmful requests presented. The model is susceptible not only to basic linguistic manipulations but also to more sophisticated AI-generated exploits. This complete failure to withstand even automated adversarial attacks underscores a profound vulnerability as noted in Rohan Paul’s Feb 4 newsletter.
Lagging Performance in Multi-Turn Conversations: Scale AI’s newly introduced MultiChallenge benchmark directly addresses the limitations of existing LLM evaluations, which have become saturated, with top models achieving near-perfect scores on simpler tasks. MultiChallenge focuses on evaluating crucial aspects of realistic, multi-turn conversations, including instruction retention across turns, inference memory, versioned editing, and self-coherence. DeepSeek-R1 finished in 10th place, falling significantly behind the top performer, Claude 3.5 Sonnet, which achieved a score of only 41.4%. This benchmark, utilizing a hybrid approach of AI-generated and human-expert refined conversations and a novel LLM-as-judge evaluation system with high agreement with human raters (93%), highlights a clear gap in DeepSeek R1’s ability to manage the nuances and complexities of sustained, dynamic dialogues, once again, as noted in Rohan Paul’s Feb 7 newsletter.
In Case You Missed This
DiLoCO, Smarter Distributed Training: Training large AI models is computationally intensive and communication bottlenecked when distributed across multiple machines. DeepMind Researchers have developed Streaming DiLoCO, “lets people distribute training of billion-scale parameters” as reported by Jack Clark. Instead of synchronizing all model parameters after each training step (which is very bandwidth-heavy), DiLoCO synchronizes only subsets of parameters sequentially. This, combined with allowing workers to continue training during synchronization and quantizing the data exchanged, significantly reduces communication overhead. Though Streaming DiLoCo uses full precision (FP32) for computing gradients, they use low-precision (4 bit) for sharing the outer gradients for the updates, with no performance decrease. This could pave the way for understanding how such methods can scaled across hyper-parameters such as model size, overtraining factor, replicas etc, allowing for heterogeneous devices usage for model training.
Nathan Lambert's recent article, dives into the implications of the broader open vs. closed AI debate. Lambert highlights arguments from both Meta's Mark Zuckerberg and DeepSeek's CEO Liang Wenfeng, both pointing towards the strategic importance of open-source AI, albeit from American and Chinese perspectives respectively. Zuckerberg argues for open-source AI becoming an "American standard" for national advantage, while Wenfeng emphasizes China's commitment to contributing to global innovation and pushing the technological frontier through open models. He suggests that for open-source AI to truly thrive, innovative feedback loops and advancements in fine-tuning are needed to make open models demonstrably more useful. Restricting open-source AI is a losing proposition and that Western governments should actively invest in open research and public-sector coalitions to ensure a vibrant and competitive open AI ecosystem, rather than focusing on restrictive measures.
Metaprompting guide - With Chain-of-Thought and Mixture-of-Experts being widely productionized in user-facing applications, I want to share this guide on meta-prompting to allow users to better leverage their LLMs.
This week, I want to share Tree of Thoughts - a strategy that moves beyond linear, single-path reasoning which Instead of generating just one thought at a time explores a tree of potential reasoning paths. This involves breaking down a problem into intermediate thoughts, generating multiple diverse thoughts at each step, evaluating these thoughts for their potential, and then strategically exploring the most promising branches. Try this template for a LLM next time before prompting it:
**Instructions:**
1. **Thought Generation (Branching): You are an expert who is consulting at least 3 distinct experts, starting at different points for the same root question. For each starting point, write a brief "thought" - a sentence or two capturing the core idea. These are the initial branches of our thought tree.
2. **Evaluation & Selection:** Review your initial thought branches and now go a layer deeper into the analysis by fulling experts in those specific domain. Select the most promising thought branches to develop further and explain why you chose them
3. **Expansion (Deeper Thoughts):** Finally, go a layer deeper for each of your selected thought branches to generate **subsequent thoughts**.
4. **Comparison & Synthesis (Choosing the Best Path):** Compare the developed thought branches and once you verify them independently, give a comprehensive answer.
Sources: