<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Programming on In Pursuit of Simplicity</title>
    <link>https://ramsayleung.github.io/en/tags/programming/</link>
    <description>Recent content in Programming on In Pursuit of Simplicity</description>
    <image>
      <title>In Pursuit of Simplicity</title>
      <url>https://ramsayleung.github.io/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E</url>
      <link>https://ramsayleung.github.io/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E</link>
    </image>
    <generator>Hugo -- 0.146.7</generator>
    <language>en</language>
    <copyright>See this site&amp;rsquo;s source code here, licensed under GPLv3 ·</copyright>
    <lastBuildDate>Mon, 10 Nov 2025 22:51:59 -0800</lastBuildDate>
    <atom:link href="https://ramsayleung.github.io/en/tags/programming/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>A Telegram Spam Blocker Bot Based On Bayesian Algorithm</title>
      <link>https://ramsayleung.github.io/en/post/2025/a_telegram_spam_blocker_bot_based_on_bayesian/</link>
      <pubDate>Sat, 30 Aug 2025 10:34:00 -0700</pubDate>
      <guid>https://ramsayleung.github.io/en/post/2025/a_telegram_spam_blocker_bot_based_on_bayesian/</guid>
      <description>&lt;h2 id=&#34;preface&#34;&gt;&lt;span class=&#34;section-num&#34;&gt;1&lt;/span&gt; Preface&lt;/h2&gt;
&lt;p&gt;I spent a weekend building a Telegram spam blocker bot based on Bayesian Algorithm &lt;code&gt;@BayesSpamSniperBot&lt;/code&gt; (&lt;a href=&#34;https://t.me/BayesSpamSniperBot&#34;&gt;https://t.me/BayesSpamSniperBot&lt;/a&gt;). The project is open-sourced at: &lt;a href=&#34;https://github.com/ramsayleung/bayes_spam_sniper&#34;&gt;https://github.com/ramsayleung/bayes_spam_sniper&lt;/a&gt;&lt;/p&gt;
&lt;h3 id=&#34;telegram&#34;&gt;&lt;span class=&#34;section-num&#34;&gt;1.1&lt;/span&gt; Telegram&lt;/h3&gt;
&lt;p&gt;Telegram is a popular instant messaging application, similar to Snapchat and WhatsApp, with over 1 billion users.&lt;/p&gt;
&lt;p&gt;It supports many powerful features like cloud chat history storage, clients for Linux, Mac, Windows, Android, IOS, and Web (all open-source), Channel, and arguably the most powerful bot system I&amp;rsquo;ve ever seen.&lt;/p&gt;</description>
      <content:encoded><![CDATA[<h2 id="preface"><span class="section-num">1</span> Preface</h2>
<p>I spent a weekend building a Telegram spam blocker bot based on Bayesian Algorithm <code>@BayesSpamSniperBot</code> (<a href="https://t.me/BayesSpamSniperBot">https://t.me/BayesSpamSniperBot</a>). The project is open-sourced at: <a href="https://github.com/ramsayleung/bayes_spam_sniper">https://github.com/ramsayleung/bayes_spam_sniper</a></p>
<h3 id="telegram"><span class="section-num">1.1</span> Telegram</h3>
<p>Telegram is a popular instant messaging application, similar to Snapchat and WhatsApp, with over 1 billion users.</p>
<p>It supports many powerful features like cloud chat history storage, clients for Linux, Mac, Windows, Android, IOS, and Web (all open-source), Channel, and arguably the most powerful bot system I&rsquo;ve ever seen.</p>
<h2 id="origin"><span class="section-num">2</span> Origin</h2>
<p>I usually listen to podcasts while running and cooking. 《<a href="https://podcasts.apple.com/us/podcast/%E8%BD%AF%E4%BB%B6%E9%82%A3%E4%BA%9B%E4%BA%8B%E5%84%BF/id1147186605">软件那些事儿(A podcast in Chinese about history and story behind software)</a>》<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> is one of my favorites, hosted by <a href="https://liuyandong.com/sample-page">栋哥</a> <sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>. Because I enjoyed 栋哥&rsquo;s show, I took the chance to join his Telegram channel.</p>
<p>栋哥&rsquo;s Telegram channel <a href="https://t.me/huruanhuying">汗牛充栋</a> <sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup> is primarily used for releasing podcast information.
He once enabled the comment for channel, but it unexpectedly attracted a flood of crypto-related users posting spam, leading him to disable comments:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-be4f5" hidden>
    <label for="zoomCheck-be4f5">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/spam_concert.jpg"/> 
    
    
    </label>
</figure>

<p>Another channel I subscribe, <a href="https://t.me/kaedeharakazuha17">Ray Tracing</a> <sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup>, also complained about the crypto spam:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-d541b" hidden>
    <label for="zoomCheck-d541b">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/ray_tracing_spam.jpg"/> 
    
    
    </label>
</figure>

<h2 id="hackers-and-painters"><span class="section-num">3</span> Hackers &amp; Painters</h2>
<p>Most common Telegram spam blocker bots are keyword-based, blocking messages by matching keywords, which can be easily bypassed by spammers.</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-1401f" hidden>
    <label for="zoomCheck-1401f">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/keyword_based_blocker.jpg"/> 
    
    
    </label>
</figure>

<p>If the messages get bypassed, it could only be deleted manually by administrator.</p>
<p>This reminded me of the situation Paul Graham described in his 2002 essay within &ldquo;Hackers &amp; Painters&rdquo;:</p>
<p>When email became popular, there was also a lot of spam. Common spam blockers were keyword matching + email address blacklists, but these were inefficient and easily circumvented.</p>
<p>Paul Graham creatively used Bayesian Theorem to implement a <a href="https://paulgraham.com/spam.html">spam blocker</a> <sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup>, and the results were surprisingly effective.</p>
<p>Isn&rsquo;t this a similar problem for Telegram spam?</p>
<p>Couldn&rsquo;t I use a similar solution to tackle Telegram spam?</p>
<h3 id="bayes-theorem"><span class="section-num">3.1</span> Bayes&rsquo; Theorem</h3>
<p>When it comes to probabilistic algorithms, the most classic example is the &ldquo;coin toss&rdquo; – a case of classical probability where each toss is an independent event, and the previous outcome doesn&rsquo;t affect the next probability.</p>
<p>However, many real-world scenarios aren&rsquo;t infinitely repeatable like coin tosses, and events are often not independent.</p>
<p>This is where Bayes Theorem shows its unique value.</p>
<p>It used to update our degree of belief in a hypothesis given certain evidence.</p>
<p>In other words, the Bayes algorithm can dynamically adjust the estimated probability of an event occurring based on continuously emerging new evidence.</p>
<p>Simply put, it&rsquo;s like the human brain&rsquo;s learning process: we start with a preliminary understanding, then revise our original view based on new information, thereby adjusting our next actions.</p>
<p>Paul Graham used Bayes theorem to continuously classify new emails as spam or not based on emails already identified as spam or non-spam (ham).</p>
<p>To understand Bayes Theorem more intuitively, I recommend this clear and easy-to-understand videos:</p>
<ul>
<li>《<a href="https://www.youtube.com/watch?v=HZGCoVF3YvM">Bayes theorem, the geometry of changing beliefs</a>》<sup id="fnref:6"><a href="#fn:6" class="footnote-ref" role="doc-noteref">6</a></sup></li>
</ul>
<h2 id="architecture-design"><span class="section-num">4</span> Architecture Design</h2>
<p>Telegram Bot supports two modes of interacting with Telegram servers:</p>
<ol>
<li>
<p>Webhook: Telegram servers actively callback a URL previously registered by the Bot whenever the Bot receives a new message. The Bot Server only needs to handle the callback messages.</p>
</li>
<li>
<p>Long Polling: The Bot Server continuously polls the Telegram servers to check for new messages and processes them if any. This bot uses this mode.</p>

    <figure>
        
        
        <input type="checkbox" id="zoomCheck-ca97a" hidden>
        <label for="zoomCheck-ca97a">
        
        
        <img class="zoomCheck" loading="lazy" src="/ox-hugo/webhook_vs_long_polling.jpg"/> 
        
        
        </label>
    </figure>

</li>
</ol>
<h4 id="message-analysis"><span class="section-num">4.0.1</span> Message Analysis</h4>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-e2518" hidden>
    <label for="zoomCheck-e2518">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/spam_analyze.jpg"/> 
    
    
    </label>
</figure>

<p>After the Bot Server receives a message, it dispatches it to a separate <code>telegram_bot_worker</code> for processing. Based on the pre-trained model, it judges whether it&rsquo;s a spam. If it is, it calls the Bot API to delete the message.</p>
<h4 id="ban-and-train"><span class="section-num">4.0.2</span> Ban and Train</h4>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-04d7c" hidden>
    <label for="zoomCheck-04d7c">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/mark_spam_and_ban_user.jpg"/> 
    
    
    </label>
</figure>

<p>After the Bot Server receives a message, it dispatches it to a separate <code>telegram_bot_worker</code> for processing. The <code>telegram_bot_worker</code> calls the bot API to delete the message and ban the user, and inserts a training data record marked as spam.</p>
<p>Saving the training data triggers a hook, creating a training message delivered to the <code>training</code> message queue. Another worker, <code>classifier_trainer</code>, subscribes to <code>training</code> messages and uses the new messages to retrain and update the model.</p>
<p>Using a queue and a background process (<code>classifier_trainer</code>) for training tasks, instead of directly using the <code>telegram_bot_worker</code>, primarily decouples the Bot request handling from model training. Otherwise, as the model size increases, training time would get longer and longer, leading to increasingly long response times.</p>
<p>Decoupling makes it easy to scale.</p>
<h2 id="why-rails"><span class="section-num">5</span> Why Rails</h2>
<p>whoever have seen my project source code might wonder, why was it implemented using Ruby on Rails?</p>
<p>Because I work with JVM languages (Java/Kotlin/Scala) and Rust, I&rsquo;m quite familiar with Java/Rust. Initially, thinking model training might require high performance, my first <a href="https://gist.github.com/ramsayleung/5848af0177a70a01d41f624e361b1b5d">prototype</a> <sup id="fnref:7"><a href="#fn:7" class="footnote-ref" role="doc-noteref">7</a></sup> was implemented in Rust, taking about half an hour.</p>
<p>But when I wanted to expand the prototype into a Telegram bot, I found I needed to handle a lot of logic related to bot interaction, mainly involving API and database operations, most of which were unrelated to the model. So, I thought of Ruby on Rails again.</p>
<p>For a single engineer building a product prototype, in my personal opinion, there&rsquo;s really no framework more efficient than <code>Ruby on Rails</code>, so I switched to Ruby on Rails.</p>
<p>New features in Rails 8 move it further towards being a so-called &ldquo;one-person full-stack framework,&rdquo; with built-in support for <code>Solid Queue</code> via the relational database.</p>
<p>The queue and background process from the architecture design were implemented with just a few lines of code, without even needing extra configuration. If the queue doesn&rsquo;t exist, the framework creates it automatically:</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span><span class="lnt">5
</span><span class="lnt">6
</span><span class="lnt">7
</span><span class="lnt">8
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-ruby" data-lang="ruby"><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">ClassifierTrainerJob</span> <span class="o">&lt;</span> <span class="no">ApplicationJob</span>
</span></span><span class="line"><span class="cl">  <span class="c1"># Job to train classifier asynchronously</span>
</span></span><span class="line"><span class="cl">  <span class="n">queue_as</span> <span class="ss">:training</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="k">def</span> <span class="nf">perform</span><span class="p">(</span><span class="n">group_id</span><span class="p">,</span> <span class="n">group_name</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="no">SpamClassifierService</span><span class="o">.</span><span class="n">rebuild_for_group</span><span class="p">(</span><span class="n">group_id</span><span class="p">,</span> <span class="n">group_name</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="k">end</span>
</span></span><span class="line"><span class="cl"><span class="k">end</span>
</span></span></code></pre></td></tr></table>
</div>
</div><p>Thanks to Rails&rsquo; powerful ORM framework and its built-in lifecycle hooks, the code to trigger the background process for retraining the model after inserting new training data is also just a few lines:</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-ruby" data-lang="ruby"><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">TrainedMessage</span> <span class="o">&lt;</span> <span class="no">ApplicationRecord</span>
</span></span><span class="line"><span class="cl">  <span class="c1"># Automatically train classifier after creating/updating a message</span>
</span></span><span class="line"><span class="cl">  <span class="n">after_create</span> <span class="ss">:retrain_classifier</span>
</span></span><span class="line"><span class="cl">  <span class="n">after_destroy</span> <span class="ss">:retrain_classifier</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="k">def</span> <span class="nf">retrain_classifier</span>
</span></span><span class="line"><span class="cl">    <span class="c1"># For efficiency, we could queue this as a background job</span>
</span></span><span class="line"><span class="cl">    <span class="no">ClassifierTrainerJob</span><span class="o">.</span><span class="n">perform_later</span><span class="p">(</span><span class="n">group_id</span><span class="p">,</span> <span class="n">group_name</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="k">end</span>
</span></span><span class="line"><span class="cl"><span class="k">end</span>
</span></span></code></pre></td></tr></table>
</div>
</div><p>Empowered by Rails&rsquo; various built-in powerful tools, I implemented the entire bot&rsquo;s functionality in just one day.</p>
<p>Seeing this, some friends might worry about performance, thinking Ruby&rsquo;s performance isn&rsquo;t great, and it&rsquo;s a dynamic language, making it hard to maintain.</p>
<p>My view remains the same as in my previous blog post 《<a href="https://ramsayleung.github.io/zh/post/2024/%E7%BC%96%E7%A8%8B%E5%8D%81%E5%B9%B4%E7%9A%84%E6%84%9F%E6%82%9F/">Thoughts on a Decade of Programming(In Chinese)</a>》<sup id="fnref:8"><a href="#fn:8" class="footnote-ref" role="doc-noteref">8</a></sup>:</p>
<p><strong><strong>Get it running first</strong></strong>.</p>
<p>Build a prototype and get it running. See if users are willing to use your product first.</p>
<p>When running speed becomes a bottleneck, your business must be very successful, and you&rsquo;ll surely have enough resources to hire a team of programmers to optimize the project into Rust/C++, or even assembly.</p>
<p>Without users, discussing performance is a pseudo-proposition.</p>
<p>As for the saying &ldquo;dynamic languages are fast in the moment, but maintaining the code is a nightmare&rdquo; I quite agree with that too.</p>
<p>Therefore, I won&rsquo;t consider dynamic languages when choosing a tech stack for a team; I would only use compiled languages, even strong-typed ones like Rust.</p>
<p>But right now, it&rsquo;s just me building a prototype, so I use whatever I&rsquo;m most proficient with.</p>
<h3 id="vibe-coding"><span class="section-num">5.1</span> Vibe Coding?</h3>
<p>Concepts like Vibe Coding and AI programming are everywhere, overwhelming the discourse. You might naturally wonder if this project was generated by Vibe Coding.</p>
<p>The answer is, I tried for a few hours and then gave up entirely. I tried both Claude 4 and Gemini 2.5 Pro.</p>
<p>I started with a Rust + Cloudflare Worker tech stack. Rust + Cloudflare Worker is a relatively niche field with limited training data. The code generated by Vibe Coding failed to compile.</p>
<p>Later, I switched to Ruby on Rails, and the problems became even worse. Ruby is a dynamic language; its syntax is almost like English, and Rails has many &ldquo;black magic&rdquo; metaprogramming features.</p>
<p>So errors only appeared at runtime. The development time saved by code generation was entirely consumed by the debugging process.</p>
<p>Another issue is that code generated by Vibe Coding often lacks design. For example, it tightly coupled the <code>Classifier</code> and <code>TrainedMessage</code> classes, having the <code>Classifier</code> persist <code>TrainedMessage</code> instances.</p>
<p>It also directly performed synchronous model training within the <code>telegram_bot_worker</code> process upon receiving training data, waiting for training to finish before returning the command result, completely neglecting to decouple receiving training data from model training.</p>
<p>One can only say that Vibe Coding is quite suitable for strongly-typed, compiled languages like Rust – at least the generated code has to compile.</p>
<p>As for those claims of &ldquo;making an APP without writing/changing a single line of code,&rdquo; I can&rsquo;t help but wonder:</p>
<p>Is the code so good that it doesn&rsquo;t need a single change? Or can the developer not identify the crux of the problem, and thus doesn&rsquo;t change anything?</p>
<h2 id="design-philosophy"><span class="section-num">6</span> Design Philosophy</h2>
<p>After developing the prototype and making the bot&rsquo;s core functionality usable, many ideas popped into my head.</p>
<p>I immediately rushed to add them to the bot,
resulting in support for nearly ten commands, plus different modes for private chats and group chats.</p>
<p>But I felt something was off. Adding so many features felt like those all-in-one super apps common in China. I began to question:</p>
<p>Would users really use all these features? Would <strong>any</strong> users use these features? Don&rsquo;t too many features also create extra cognitive burden?</p>
<p>My favorite ad blocker, <a href="https://ublockorigin.com/">Ublock Origin</a> <sup id="fnref:9"><a href="#fn:9" class="footnote-ref" role="doc-noteref">9</a></sup>, is powerful and extremely effective at blocking, yet very simple and easy to use.</p>
<p>Recalling the design philosophy mentioned in 《<a href="https://ramsayleung.github.io/zh/post/2025/a_philosophy_of_software_design/">A Philosophy of Software Design(My thought about the book in Chinese)</a>》<sup id="fnref:10"><a href="#fn:10" class="footnote-ref" role="doc-noteref">10</a></sup>, the interface should be simple and easy to use, even if the functionality underneath is complex and rich.</p>
<p>So, I first removed all commands I considered unrelated to the core functionality.</p>
<p>Furthermore, considering that most users might not have a technical background and might not know how to use commands, I optimized the interface to use buttons as much as possible, allowing users to click directly, improving usability:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-5f8d5" hidden>
    <label for="zoomCheck-5f8d5">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/start_en.jpg"/> 
    
    
    </label>
</figure>


<figure>
    
    
    <input type="checkbox" id="zoomCheck-bc48d" hidden>
    <label for="zoomCheck-bc48d">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/help_en.jpg"/> 
    
    
    </label>
</figure>

<p>I also wanted to support multiple languages (e.g., automatically switching to Chinese or English based on the user&rsquo;s system language). This required decent Internationalization support.</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-e207f" hidden>
    <label for="zoomCheck-e207f">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/start_zh.jpg"/> 
    
    
    </label>
</figure>


<figure>
    
    
    <input type="checkbox" id="zoomCheck-ca4b6" hidden>
    <label for="zoomCheck-ca4b6">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/help_page_zh.jpg"/> 
    
    
    </label>
</figure>

<p>Over 60% of the code in the core service class <a href="https://github.com/ramsayleung/bayes_spam_sniper/blob/master/app/services/telegram_botter.rb">telegram_botter.rb</a> was introduced for such usability improvements.</p>
<p>Simplicity for the user, complexity for the developer.</p>
<h3 id="how-to-use"><span class="section-num">6.1</span> How to Use</h3>
<p>Just two steps, and the bot works automatically.</p>
<ul>
<li><a href="https://t.me/BayesSpamSniperBot?startgroup=true">Add the bot (@BayesSpamSniperBot) to your group</a></li>
<li>Grant the bot admin permissions (Delete messages, Ban users)</li>
</ul>
<p>After these two steps, the bot will not only start working automatically, identifying spam within the group, deleting text messages, and banning users who send spam more than 3 times;</p>
<p>It will also become smarter with community usage (via <code>/markspam</code> and <code>/feedspam</code>).</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-28a2c" hidden>
    <label for="zoomCheck-28a2c">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/detect_spam_and_ban_user.jpg"/> 
    
    
    </label>
</figure>

<p>The design philosophy of this bot is to minimize disruption to admins and users, provide simple operation commands, and maximize automation.
Therefore, this bot only offers the following three commands (supporting auto-completion with &ldquo;/&rdquo;):</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-ecf1f" hidden>
    <label for="zoomCheck-ecf1f">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/command_auto_completion.jpg"/> 
    
    
    </label>
</figure>

<h4 id="markspam"><span class="section-num">6.1.1</span> <code>/markspam</code></h4>
<p>Delete spam messages and ban the user. Requires admin permissions.</p>
<p>Reply <code>/markspam</code> to the message you want to ban, and the bot will automatically delete that message and ban the user.</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-14d59" hidden>
    <label for="zoomCheck-14d59">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/markspam_2.jpg"/> 
    
    
    </label>
</figure>

<p>(Message has also been deleted)
<img loading="lazy" src="/ox-hugo/markspam.jpg"></p>
<p>Unlike common group management bots, this command not only deletes the spam and bans the user but also, because this message is marked as spam by an admin with very high confidence, uses this spam ad as training data to update the model in real-time.</p>
<p>Similar messages will not only be identified next time, but all groups using this bot will benefit, as similar texts will also be marked as spam.</p>
<h4 id="listspam"><span class="section-num">6.1.2</span> <code>/listspam</code></h4>
<p>View the list of spam messages. Requires admin permissions.</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-d0038" hidden>
    <label for="zoomCheck-d0038">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/listspam.jpg"/> 
    
    
    </label>
</figure>

<p>View the list of spam messages and proactively mark false positive spam as normal.</p>
<h4 id="listbanuser"><span class="section-num">6.1.3</span> <code>/listbanuser</code></h4>
<p>View the list of banned accounts. Requires admin permissions.</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-7751e" hidden>
    <label for="zoomCheck-7751e">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/listbanuser.jpg"/> 
    
    
    </label>
</figure>

<p>View the list of banned users and proactively unban them.</p>
<h4 id="feedspam"><span class="section-num">6.1.4</span> <code>/feedspam</code></h4>
<p>Feed spam messages for training. No permissions required. Can be used in private chat or in-group.</p>
<p>Feeding in private chat:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-2a139" hidden>
    <label for="zoomCheck-2a139">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/feedspam.jpg"/> 
    
    
    </label>
</figure>

<p>Feeding in-group:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-1fc12" hidden>
    <label for="zoomCheck-1fc12">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/feedspam2.jpg"/> 
    
    
    </label>
</figure>

<h2 id="eating-your-own-dog-food"><span class="section-num">7</span> Eating your own dog food</h2>
<p>In the software development field, there&rsquo;s a saying: &ldquo;Eating your own dog food,&rdquo; which means you should use the things you develop yourself.</p>
<p>So I created my own channel for testing: <a href="https://t.me/pipeapplebun">菠萝油与天光墟</a> <sup id="fnref:11"><a href="#fn:11" class="footnote-ref" role="doc-noteref">11</a></sup>. Unfortunately, it has very few subscribers,
which fails to attract many spammers. So everyone is welcome to subscribe or come in to post spam, to attract more spammers.</p>
<p>In my channel, everyone has the right to speak freely :-) (the only slight drawback is a limit on frequency ).</p>
<p>Since no one was posting spam in my channel, and suffering from a lack of training data, I had to do it in the hard way: to join various crypto groups, NSFW groups, and actively seek out spam:</p>
<p>The screenshots are Chinese Telegram group contains a lot of spammers and spam:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-3779e" hidden>
    <label for="zoomCheck-3779e">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/telegram_group1.jpg"/> 
    
    
    </label>
</figure>


<figure>
    
    
    <input type="checkbox" id="zoomCheck-35262" hidden>
    <label for="zoomCheck-35262">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/telegram_group2.jpg"/> 
    
    
    </label>
</figure>


<figure>
    
    
    <input type="checkbox" id="zoomCheck-b0b9b" hidden>
    <label for="zoomCheck-b0b9b">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/spam_sample.jpg"/> 
    
    
    </label>
</figure>

<p>Since developing this bot, my perspective on spam has changed. I used to find spam annoying in other groups, but now I&rsquo;m happy to see them in other groups,
as they are valuable training data that I need to record quickly before they get deleted.</p>
<h3 id="the-ingenuity-of-spam"><span class="section-num">7.1</span> The Ingenuity of Spam</h3>
<p>The algorithm&rsquo;s effectiveness in other people&rsquo;s stories is always surprisingly good, but when I run it myself, I always find various uncovered cases and unexpected surprises, just like life.</p>
<p>Although keyword blocker is inefficient, the spam we actually see are those that have already bypassed keyword filters.</p>
<p>For example:</p>
<blockquote>
<p>在 币圈 想 赚 钱，那 你 不关 注 这 个 王 牌 社 区，真的太可惜了，真 心 推 荐，每 天 都 有 免 费 策 略
(Want to make money in the crypto circle? It&rsquo;s a real pity if you don&rsquo;t follow this ace community. Sincerely recommended, free strategies every day)</p></blockquote>
<p>Or</p>
<blockquote>
<p>这人简-介挂的 合-约-报单群组挺牛的ETH500点，大饼5200点！ + @BTCETHl6666
(The contract reporting group linked in this person&rsquo;s bio is pretty awesome. ETH to $500, BTC to $5200! + @BTCETHl6666)</p></blockquote>
<p>The former uses spaces to bypass keywords, the latter uses punctuation.</p>
<p>Unlike languages based on the Latin alphabet which naturally use spaces for word separation, Chinese requires word segmentation before statistical analysis with Bayes&rsquo; theorem.</p>
<blockquote>
<p>the fox jumped over the lazy dog</p>
<p>我们的中文就不一样了(Our Chinese is different)</p></blockquote>
<p>&ldquo;我们的中文就不一样了&rdquo; (Our Chinese is different) would be segmented into &ldquo;我们 | 的 | 中文 | 就 | 不 | 一样 | 了&rdquo; before word frequency can be counted.</p>
<p>But for the spam <code>在 币圈 想 赚 钱，那 你 不关 注 这 个 王 牌 社 区，真的太可惜了，真 心 推 荐，每 天 都 有 免 费 策 略</code>, the spaces not only affect keyword matching but also affect segmentation. The segmentation result for this sentence becomes(incorrect):</p>
<p><code>在 | | 币圈 | | 想 | | 赚 | | 钱 | ， | 那 | | 你 | | 不 | 关 | | 注 | | 这 | | 个 | | 王 | | 牌 | | 社 | | 区 | ， | 真的 | 太 | 可惜 | 了 | ， | 真 | | 心 | | 推 | | 荐 | ， | 每 | | 天 | | 都 | | 有 | | 免 | | 费 | | 策 | | 略</code></p>
<p><code>这人简-介挂的 合-约-报单群组挺牛的ETH500点，大饼5200点！ + @BTCETHl6666</code> would be segmented into:</p>
<p><code>这人简 | - | 介挂 | 的 | | 合 | - | 约 | - | 报单 | 群组 | 挺 | 牛 | 的 | ETH500 | 点 | ， | 大饼 | 5200 | 点 | ！ | | + | | @ | BTCETHl6666</code></p>
<p>Unsanitized training data would affect the model&rsquo;s results, showing the importance of training data quality. Therefore, I performed corresponding preprocessing on the training data:</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span><span class="lnt">11
</span><span class="lnt">12
</span><span class="lnt">13
</span><span class="lnt">14
</span><span class="lnt">15
</span><span class="lnt">16
</span><span class="lnt">17
</span><span class="lnt">18
</span><span class="lnt">19
</span><span class="lnt">20
</span><span class="lnt">21
</span><span class="lnt">22
</span><span class="lnt">23
</span><span class="lnt">24
</span><span class="lnt">25
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-ruby" data-lang="ruby"><span class="line"><span class="cl"><span class="c1"># Step 1: Handle anti-spam separators</span>
</span></span><span class="line"><span class="cl"><span class="c1"># This still handles the cases like &#34;合-约&#34; -&gt; &#34;合约&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">previous</span> <span class="o">=</span> <span class="s2">&#34;&#34;</span>
</span></span><span class="line"><span class="cl"><span class="k">while</span> <span class="n">previous</span> <span class="o">!=</span> <span class="n">cleaned</span>
</span></span><span class="line"><span class="cl">  <span class="n">previous</span> <span class="o">=</span> <span class="n">cleaned</span><span class="o">.</span><span class="n">dup</span>
</span></span><span class="line"><span class="cl">  <span class="n">cleaned</span> <span class="o">=</span> <span class="n">cleaned</span><span class="o">.</span><span class="n">gsub</span><span class="p">(</span><span class="sr">/([一-龯A-Za-z0-9])[^一-龯A-Za-z0-9]+([一-龯A-Za-z0-9])/</span><span class="p">,</span> <span class="s1">&#39;\1\2&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="k">end</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Step 2: Handle anti-spam SPACES between Chinese characters</span>
</span></span><span class="line"><span class="cl"><span class="c1"># This specifically targets the &#34;想 赚 钱&#34; -&gt; &#34;想赚钱&#34; case.</span>
</span></span><span class="line"><span class="cl"><span class="c1"># We run it in a loop to handle multiple spaces, e.g., &#34;社 区&#34; -&gt; &#34;社区&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">previous</span> <span class="o">=</span> <span class="s2">&#34;&#34;</span>
</span></span><span class="line"><span class="cl"><span class="k">while</span> <span class="n">previous</span> <span class="o">!=</span> <span class="n">cleaned</span>
</span></span><span class="line"><span class="cl">  <span class="n">previous</span> <span class="o">=</span> <span class="n">cleaned</span><span class="o">.</span><span class="n">dup</span>
</span></span><span class="line"><span class="cl">  <span class="c1"># Find a Chinese char, followed by one or more spaces, then another Chinese char</span>
</span></span><span class="line"><span class="cl">  <span class="n">cleaned</span> <span class="o">=</span> <span class="n">cleaned</span><span class="o">.</span><span class="n">gsub</span><span class="p">(</span><span class="sr">/([一-龯])(\s+)([一-龯])/</span><span class="p">,</span> <span class="s1">&#39;\1\3&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="k">end</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Step 3: Add strategic spaces</span>
</span></span><span class="line"><span class="cl"><span class="c1"># This helps jieba segment properly, e.g., &#34;社区ETH&#34; -&gt; &#34;社区 ETH&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">cleaned</span> <span class="o">=</span> <span class="n">cleaned</span><span class="o">.</span><span class="n">gsub</span><span class="p">(</span><span class="sr">/([一-龯])([A-Za-z0-9])/</span><span class="p">,</span> <span class="s1">&#39;\1 \2&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">cleaned</span> <span class="o">=</span> <span class="n">cleaned</span><span class="o">.</span><span class="n">gsub</span><span class="p">(</span><span class="sr">/([A-Za-z0-9])([一-龯])/</span><span class="p">,</span> <span class="s1">&#39;\1 \2&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Step 4: Remove excessive space</span>
</span></span><span class="line"><span class="cl"><span class="n">cleaned</span> <span class="o">=</span> <span class="n">cleaned</span><span class="o">.</span><span class="n">gsub</span><span class="p">(</span><span class="sr">/\s+/</span><span class="p">,</span> <span class="s1">&#39; &#39;</span><span class="p">)</span><span class="o">.</span><span class="n">strip</span>
</span></span></code></pre></td></tr></table>
</div>
</div><p>After preprocessing, <code>在 币圈 想 赚 钱，那 你 不关 注 这 个 王 牌 社 区，真的太可惜了，真 心 推 荐，每 天 都 有 免 费 策 略</code> becomes <code>在币圈想赚钱那你不关注这个王牌社区真的太可惜了真心推荐每天都有免费策略</code> (Note: legitimate commas are also removed here. I find the segmentation result acceptable compared to the impact of excessive punctuation). Its segmentation is:</p>
<p><code>在 | 币圈 | 想 | 赚钱 | 那 | 你 | 不 | 关注 | 这个 | 王牌 | 社区 | 真的 | 太 | 可惜 | 了 | 真心 | 推荐 | 每天 | 都 | 有 | 免费 | 策略</code></p>
<p><code>这人简-介挂的 合-约-报单群组挺牛的ETH500点，大饼5200点！ + @BTCETHl6666</code> becomes <code>这人简介挂的合约报单群组挺牛的 ETH500 点大饼 5200 点！ + @BTCETHl6666</code>. Its segmentation is:</p>
<p><code>这 | 人 | 简介 | 挂 | 的 | 合约 | 报单 | 群组 | 挺 | 牛 | 的 | | ETH500 | | 点 | 大饼 | | 5200 | | 点 | ！ | | + | | @ | BTCETHl6666</code></p>
<h4 id="genius-new-tricks"><span class="section-num">7.1.1</span> Genius New Tricks</h4>
<p>Seeing many spam, I has to admire the creativity of spammers.</p>
<p>Because sending spam in messages gets caught by blockers, they innovatively came up with new tricks:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-fe0e1" hidden>
    <label for="zoomCheck-fe0e1">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/spam_by_username.jpg"/> 
    
    
    </label>
</figure>

<p>The messages contain normal text, but the profile picture and username are spam. This way, the spam blocker can&rsquo;t work. Truly ingenious.</p>
<p>Faced with such creative opponents, I adapted by building a training model for usernames. During detection, both the message text model and the username model are checked.
If either one considers it spam, it gets blocked.</p>
<p>One could go further by doing OCR on profile pictures to extract text and add another model for profile pictures, but OCR is quite costly, so I&rsquo;ll hold off for now.</p>
<h3 id="optimization"><span class="section-num">7.2</span> Optimization</h3>
<p>Without users, any optimization is unnecessary, as premature optimization is the root of all evil.
Therefore, I focus on building the prototype first. But this doesn&rsquo;t mean the prototype has no room for optimization.</p>
<p>I have several optimization ideas in mind:</p>
<ol>
<li>jieba&rsquo;s segmentation might not be the best; other NLP algorithms could be used later for optimization.</li>
<li>Retraining on every single training message is inefficient. A batching mechanism could be added, waiting for 5 minutes or accumulating 100 messages before processing.</li>
<li>Currently, the entire model is computed in memory and persisted to the DB after calculation. A cache layer between memory and the database could optimize performance.</li>
<li>The Bayesian algorithm might not be effective enough; a more complex machine learning model could be used.</li>
</ol>
<p>But these optimization points are all Good to have, not Must have. I&rsquo;ll optimize them when actual problems arise later.</p>
<h2 id="does-it-capture-spam"><span class="section-num">8</span> Does it capture spam?</h2>
<p>Sending a message using transformed spam words:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-cdd9f" hidden>
    <label for="zoomCheck-cdd9f">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/spam_messge_2.jpg"/> 
    
    
    </label>
</figure>

<p>Successfully detected and automatically deleted:</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-78c5f" hidden>
    <label for="zoomCheck-78c5f">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/deleted_spam.jpg"/> 
    
    
    </label>
</figure>

<p>Some you might say this is just a demo showcase. Why are spam posted by others in my group still not being detected?</p>
<p>Because the Bayesian algorithm is fundamentally a probability-based algorithm. If it hasn&rsquo;t encountered similar spam before, it cannot determine whether they are spam :(</p>
<p>Be patient. All you need to do is use the <code>/markspam</code> command to delete the message and ban the user. This helps train the bot, and all users of this bot will benefit.</p>
<h2 id="conclusion"><span class="section-num">9</span> Conclusion</h2>
<p>I thoroughly enjoyed this creative process: discovering a problem, having a spark of inspiration, building a prototype, and finally polishing it into a complete project.</p>
<p>Although this is purely powered by passion – the code is open source, and I have to pay for the server out of my own pocket, with no material return.</p>
<p>But every time I see the bot successfully block an spam, that joy of creation and see it&rsquo;s working is the best reward.</p>
<ul>
<li>Project repository: <a href="https://github.com/ramsayleung/bayes_spam_sniper">https://github.com/ramsayleung/bayes_spam_sniper</a></li>
<li>Try it now: <a href="https://t.me/BayesSpamSniperBot">https://t.me/BayesSpamSniperBot</a></li>
</ul>
<div class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1">
<p><a href="https://podcasts.apple.com/us/podcast/%E8%BD%AF%E4%BB%B6%E9%82%A3%E4%BA%9B%E4%BA%8B%E5%84%BF/id1147186605">https://podcasts.apple.com/us/podcast/%E8%BD%AF%E4%BB%B6%E9%82%A3%E4%BA%9B%E4%BA%8B%E5%84%BF/id1147186605</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2">
<p><a href="https://liuyandong.com/sample-page">https://liuyandong.com/sample-page</a>&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3">
<p><a href="https://t.me/huruanhuying">https://t.me/huruanhuying</a>&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:4">
<p><a href="https://t.me/kaedeharakazuha17">https://t.me/kaedeharakazuha17</a>&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:5">
<p><a href="https://paulgraham.com/spam.html">https://paulgraham.com/spam.html</a>&#160;<a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:6">
<p><a href="https://www.youtube.com/watch?v=HZGCoVF3YvM">https://www.youtube.com/watch?v=HZGCoVF3YvM</a>&#160;<a href="#fnref:6" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:7">
<p><a href="https://gist.github.com/ramsayleung/5848af0177a70a01d41f624e361b1b5d">https://gist.github.com/ramsayleung/5848af0177a70a01d41f624e361b1b5d</a>&#160;<a href="#fnref:7" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:8">
<p><a href="https://ramsayleung.github.io/zh/post/2024/%E7%BC%96%E7%A8%8B%E5%8D%81%E5%B9%B4%E7%9A%84%E6%84%9F%E6%82%9F/">https://ramsayleung.github.io/zh/post/2024/%E7%BC%96%E7%A8%8B%E5%8D%81%E5%B9%B4%E7%9A%84%E6%84%9F%E6%82%9F/</a>&#160;<a href="#fnref:8" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:9">
<p><a href="https://ublockorigin.com/">https://ublockorigin.com/</a>&#160;<a href="#fnref:9" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:10">
<p><a href="https://ramsayleung.github.io/zh/post/2025/a_philosophy_of_software_design/">https://ramsayleung.github.io/zh/post/2025/a_philosophy_of_software_design/</a>&#160;<a href="#fnref:10" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:11">
<p><a href="https://t.me/pipeapplebun">https://t.me/pipeapplebun</a>&#160;<a href="#fnref:11" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</div>
]]></content:encoded>
    </item>
    <item>
      <title>Reflections on Ten Years of Programming</title>
      <link>https://ramsayleung.github.io/en/post/2024/reflection_on_ten_years_of_programming/</link>
      <pubDate>Sun, 15 Dec 2024 21:09:00 -0800</pubDate>
      <guid>https://ramsayleung.github.io/en/post/2024/reflection_on_ten_years_of_programming/</guid>
      <description>&lt;h2 id=&#34;preface&#34;&gt;&lt;span class=&#34;section-num&#34;&gt;1&lt;/span&gt; Preface&lt;/h2&gt;
&lt;p&gt;Malcolm Gladwell&amp;rsquo;s &amp;ldquo;10,000-hour rule&amp;rdquo; suggests that continuous investment of 10,000 hours of effort is sufficient to reach expert level in any field.
Based on 20 hours of practice per week, this requires about 3 hours of daily investment, taking roughly ten years to achieve this goal.&lt;/p&gt;
&lt;p&gt;Since I wrote my first line of C code, more than ten years have passed.
During this period, I have written over 300,000 lines of code, some of which, written at WeChat, have served more than 1 billion users.&lt;/p&gt;</description>
      <content:encoded><![CDATA[<h2 id="preface"><span class="section-num">1</span> Preface</h2>
<p>Malcolm Gladwell&rsquo;s &ldquo;10,000-hour rule&rdquo; suggests that continuous investment of 10,000 hours of effort is sufficient to reach expert level in any field.
Based on 20 hours of practice per week, this requires about 3 hours of daily investment, taking roughly ten years to achieve this goal.</p>
<p>Since I wrote my first line of C code, more than ten years have passed.
During this period, I have written over 300,000 lines of code, some of which, written at WeChat, have served more than 1 billion users.</p>
<p>Despite having written so much code, I still dare not call myself an expert.</p>
<p>However, years of being a &ldquo;builder&rdquo;, coding day after day, have allowed me to accumulate considerable insights.</p>
<p>&ldquo;Practice makes perfect&rdquo; - these insights are both reflections on programming technology and experiences of professional life.</p>
<h2 id="continuous-learning"><span class="section-num">2</span> Continuous Learning</h2>
<p>Although I started programming with C in university, my main language in college was Java,
as Java is a very mature industrial language with rich frameworks, highly popular in enterprises, with a lot job opportunities.</p>
<p>I started web development with Java Servlets, then learned the very popular JavaEE enterprise development framework SSH, namely <a href="https://struts.apache.org/">Struts2</a> <sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>+ <a href="https://spring.io/projects/spring-framework">Spring</a> <sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>+ <a href="https://hibernate.org/">Hibernate</a> <sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>. Struts2 handled control logic, Spring provided decoupling, and Hibernate as ORM.</p>
<p>By the time I started job hunting, the SSH concept had changed - Struts2 was replaced by <a href="https://docs.spring.io/spring-framework/reference/web/webmvc.html">SpringMVC</a> <sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup>, and SSH became SpringMVC + Spring + Hibernate.</p>
<p>When I interned at Ant Group, I discovered that the team&rsquo;s codebase didn&rsquo;t use Hibernate for database, but rather <a href="https://ibatis.apache.org/">Ibatis</a> <sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup>, which later switched to the newer <a href="https://mybatis.org/mybatis-3/">MyBatis</a> <sup id="fnref:6"><a href="#fn:6" class="footnote-ref" role="doc-noteref">6</a></sup>.</p>
<p>Ant Group internally didn&rsquo;t use Spring/SpringMVC either, but rather their in-house <a href="https://github.com/sofastack/sofa-rpc">SOFA framework</a> <sup id="fnref:7"><a href="#fn:7" class="footnote-ref" role="doc-noteref">7</a></sup>. The Spring community later felt that the Spring framework was too heavyweight and not conducive to rapid development, so they developed the lighter <a href="https://spring.io/projects/spring-boot">SpringBoot</a> <sup id="fnref:8"><a href="#fn:8" class="footnote-ref" role="doc-noteref">8</a></sup>, while Ant internally launched the SOFA version called <a href="https://github.com/sofastack/sofa-boot">SOFABoot</a> <sup id="fnref:9"><a href="#fn:9" class="footnote-ref" role="doc-noteref">9</a></sup>.</p>
<p>After joining WeChat Pay, I initially wrote C++ using WeChat in-house RPC framework called Svrkit. Later, due to Big Data projects, I started using Spark + Python + Hive SQL.</p>
<p>Now at AWS S3, because the business has extremely high performance and resource usage requirements, I&rsquo;m using Rust again, while legacy business still uses Java. After going full circle, I&rsquo;m back on the Java path.</p>
<p>Counting them all, over these years, I&rsquo;ve written production code in Java, C++, Python, Rust, Scala, Kotlin, and JavaScript/TypeScript.</p>
<p>Beyond work, I&rsquo;ve also learned Scheme while studying <a href="https://github.com/ramsayleung/sicp_solution">SICP</a>, Emacs Lisp from using Emacs, Swift for indie development, Ruby for Ruby on Rails, Golang for load testing, and various frameworks and libraries for different languages.</p>
<p>Since I started learning programming, I&rsquo;ve learned at least half a dozen programming languages.</p>
<p>I never define myself as a programmer of any specific language, like Java programmer or C++ programmer - I just call myself a Software Development Engineer. Languages are merely tools; as long as I keep learning, I&rsquo;ll naturally learn new programming languages when encountering new scenarios.</p>
<p>The computer world changes rapidly - new frameworks might emerge every few months, and new languages become popular every few years. Continuous learning is essential for maintaining competitiveness..</p>
<h2 id="independent-thinking"><span class="section-num">3</span> Independent Thinking</h2>
<p>WeChat used to have a tradition of giving out annual gifts to employees.</p>
<p>In 2022, the annual gift we received was indeed an aluminum plate with &ldquo;Keep Independent Thinking 2022&rdquo; written on it.</p>

<figure>
    
    
    <input type="checkbox" id="zoomCheck-e312c" hidden>
    <label for="zoomCheck-e312c">
    
    
    <img class="zoomCheck" loading="lazy" src="/ox-hugo/think_independently.jpg"/> 
    
    
    </label>
</figure>

<p>The creator of WeChat has always emphasized the importance of &ldquo;independent thinking&rdquo; for WeChat, believing that if he had to choose the most important quality, he would choose &ldquo;independent thinking.&rdquo;</p>
<p>What superiors say isn&rsquo;t necessarily right, what teachers say isn&rsquo;t necessarily right, what academic institutions say isn&rsquo;t necessarily right, what media says isn&rsquo;t necessarily right, and loud voices certainly aren&rsquo;t necessarily right - after all, being reasonable doesn&rsquo;t require being loud.</p>
<p>For example, microservice architecture is very popular, and many companies are implementing microservices - does this mean monolithic architecture shouldn&rsquo;t be used?</p>
<p>Should startups or small teams adopt microservice architecture for new businesses? Or should they start with monolithic architecture and migrate to services as the business grows?</p>
<p>Development inevitably involves various decisions, such as technology selection. For your requirements, you might find a dozen components that &ldquo;seemingly&rdquo; meet the requirements,
and you might search online for evaluations of each component, finding diverse opinions. You need to independently analyze each component, identify their pros and cons, and make decisions based on your team&rsquo;s characteristics.</p>
<p>Regarding independent thinking, my favorite quote is from the HBO miniseries &ldquo;Chernobyl,&rdquo;
when scientist Valery Legasov asks the KGB to release his colleague Ulana Khomyuk, who was investigating the truth, saying he could guarantee she was no problem, the KGB chief responds:</p>
<blockquote>
<p>Trust, but verify.</p></blockquote>
<p>Because the first step of independent thinking is: start questioning.</p>
<h2 id="get-it-running-first"><span class="section-num">4</span> Get It Running First</h2>
<p>This phrase has a well-known variant: &ldquo;It&rsquo;s not like it doesn&rsquo;t work.&rdquo;</p>
<p>Many programmers are perfectionists, especially those who have read &ldquo;Refactoring&rdquo; and &ldquo;Design Patterns,&rdquo; who tend to spend a lot of time optimizing code and refactoring.</p>
<p>I used to have similar impulses, always wanting to spend time optimizing code, but after working on many projects, I have a strong feeling that it&rsquo;s better to get the MVP launch first and let users experience it early.</p>
<p>If no users are using it, even the best and most beautiful code is meaningless.</p>
<p>So when I often see community members asking what language and framework to use for side projects, whether PHP/Python/Ruby would be too slow, my view has always been: build a prototype first, find the first user first.</p>
<p>When performance becomes a bottleneck, your business is already very success, and you&rsquo;ll definitely have enough money to hire a dozen programmers to rewrite your project in Rust/C++, or assembly.</p>
<p>In this regard, I strongly agree with the my co-worker sitting next to me about code quality:</p>
<blockquote>
<p>make it run, make it fast, make it beautiful.</p></blockquote>
<p>Recently attempting side projects, I have a profound realization that technology might be the least important aspect of business.</p>
<p>Building a product from scratch and promoting it to users - users only care whether your product is useful and can solve their problems.</p>
<p>They don&rsquo;t care whether you wrote it in C++/Java or JavaScript, nor do they care whether your code is elegant. Rather than obsessing over technology selection, it&rsquo;s better to build the product first and let users try it.</p>
<h2 id="the-most-comfortable-tool-is-the-best"><span class="section-num">5</span> The Most Comfortable Tool is the Best</h2>
<p>I often see people in the community asking what&rsquo;s the best language, best framework, best editor, best operating system.</p>
<p>&ldquo;Best&rdquo; is quite a subjective conclusion, and there&rsquo;s no &ldquo;best&rdquo; solution for all scenarios, but I often see community members arguing over which language is better.</p>
<p>Or when someone shares about A, others reply that B/C/D is better, then arguments ensue.</p>
<p>This reminds me of the group identity phenomenon mentioned in &ldquo;The Social Animal,&rdquo; a famous social psychology work.</p>
<p>When fans develop strong identification with a team, they view the team as part of their self-identity, leading them to:</p>
<ol>
<li>Use &ldquo;we&rdquo; instead of &ldquo;they&rdquo; to refer to the team</li>
<li>View the team&rsquo;s success as personal success</li>
<li>React defensively to criticism of the team, viewing such criticism as attacks on themselves</li>
</ol>
<p>If someone asks me this question, I answer &ldquo;the tool you&rsquo;re most comfortable and familiar with is best.&rdquo;</p>
<p>Even if for fun, programming&rsquo;s purpose is still to use computers to solve problems, and the best tool for solving problems is the one you&rsquo;re most familiar with.</p>
<p>Unless the tool you know isn&rsquo;t suitable for your problem, then naturally you need a new tool - don&rsquo;t force a square peg into a round hole.</p>
<p>Of course, if it&rsquo;s to satisfy curiosity and learn a new language, then choose what interests you.</p>
<p>When I learned Rust in 2017, it was simply because I had no classes in senior year, plenty of time, and wanted to learn something interesting and new. Rust 1.0 had only been released for 2 years then - I never expected to find work with Rust.</p>
<p>I can&rsquo;t remember where I read this passage:</p>
<blockquote>
<p>I once asked myself similar questions:</p>
<ol>
<li>Do good things necessarily become popular? Not necessarily</li>
<li>Are things I like necessarily good things? Not necessarily</li>
<li>Will I spend time and energy on something that might not become popular but I like? Yes</li>
</ol></blockquote>
<h2 id="communicate-more-with-people"><span class="section-num">6</span> Communicate More with People</h2>
<p>While programmers certainly work with machines, fundamentally we&rsquo;re still solving human problems.</p>
<p>When I first learned programming, I had a misconception that as long as I mastered the technology, I wouldn&rsquo;t need to care about &ldquo;interpersonal relationships.&rdquo;</p>
<p>Therefore, after entering the workplace, I held such views and acted accordingly. While I wasn&rsquo;t cold to others, I inevitably was, as friends described: &ldquo;aloof.&rdquo;</p>
<p>But after working for a long time, I realized that &ldquo;interpersonal relationships&rdquo; are inevitable - which is called networking and connections.</p>
<p>Even if my technical abilities are solid, I need to be seen by others. Having good relationships with colleagues and leaders allows for &ldquo;win-win&rdquo; when achievements are made.</p>
<p>So now I usually chat with colleagues whether there&rsquo;s something specific or not, both to know each other better and learn more about the group, and to find potential optimization points from colleagues&rsquo; complaints, practicing my philosophy of &ldquo;Work hard and be nice to people.&rdquo;</p>
<p>After doing this job for a long time, I find out that software engineering is fundamentally human systems engineering.</p>
<h2 id="code-isn-t-everything"><span class="section-num">7</span> Code Isn&rsquo;t Everything</h2>
<p>After writing programs for a while, it&rsquo;s easy to have illusion that everything can be solved with code.</p>
<p>When you&rsquo;re holding a hammer, everything looks like a nail.</p>
<p>The reality I learned is that many things cannot be solved with code. Code is just a tool that can only be used in appropriate scenarios - avoid path dependency.</p>
<p>Knowing how to write code isn&rsquo;t useful, you also need to write articles, give presentations within the company, let others &ldquo;see you.&rdquo;</p>
<p>Professional ability in programming and project delivery is certainly important, but you also need soft skills in marketing yourself.</p>
<p>Chatting with the boss occasionally to increase communication and showing your visibility regularly might be more useful than launching ten projects.</p>
<h2 id="work-with-excellent-people"><span class="section-num">8</span> Work with Excellent People</h2>
<p>After years in the industry, having worked at Ant Group, WeChat, and AWS, and having worked with all kinds of colleagues, I have an increasingly strong insight:</p>
<p><strong><strong>Work with excellent people</strong></strong></p>
<p>Not only can you learn excellent qualities from them and improve technical abilities, you can learn best practices and engineering experience. During code reviews, you can learn better programming approaches, and when encountering problems, you have reliable teammates to help and guide you.</p>
<p>You&rsquo;ll understand the uniqueness of systems developed by excellent programmers, know what simple and ergonomic systems look like, and develop your own technical taste.</p>
<p>Taste and aesthetics are abstract concepts, but after using good systems, you naturally won&rsquo;t be interested in those crude, poorly made systems that rely on boss endorsement for forced promotion.</p>
<p>Improving technical taste, while enhancing our technical cognition, can in turn help us improve design capabilities.</p>
<p>Another benefit of working with excellent colleagues is building high-quality professional networks, which benefits career development and provides more options when changing jobs or switching tracks.</p>
<p>Although startups also have excellent developers, on average, Big Tech have higher proportions of excellent programmers, as they offer more competitive salaries and benefits, naturally having higher recruitment standards.</p>
<p>For example, WeChat has a so-called interview committee(like a group of bar raiser). Besides interviewers from the hiring department, candidates must also pass interviews by committee interviewers to avoid lowering standards for quick hiring.</p>
<p>So I personally suggest that new graduates should go to big tech when possible, to gain experience.</p>
<p>Although I left WeChat almost two years ago, I still miss the colleagues I worked with in the same group. They were truly technically excellent, super nice people, and willing to help.</p>
<h2 id="health-is-the-foundation-of-everything"><span class="section-num">9</span> Health Is the Foundation of Everything</h2>
<p>After programming for so many years, I&rsquo;ve developed a pile of occupational diseases.</p>
<p>I had tenosynovitis since university, developed a lower back pain after working for a few years and my once thick, dark hair is now increasingly sparse.</p>
<p>Because Tencent headquarters had a free gym, I basically went to the company gym on workdays to take advantage of company benefits - 2 days of cardio running, 2 days of anaerobic equipment training, persisting for almost 3 years.</p>
<p>I also started paying attention to my diet, trying to eat less oil and sugar and avoid alcohol.</p>
<p>While fitness isn&rsquo;t a cure-all, at least I feel charged to handle high-intensity work.</p>
<p>Only when I lose something do I learn to cherish it. Only when I start taking medication and going to hospital for follow-ups do I start paying attention to my body.</p>
<p>Although programming is fun and supporting family is important, I still need to pay attention to my body. After all, health is the foundation of everything - if it breaks down, there are no other exciting stories.</p>
<h2 id="summary"><span class="section-num">10</span> Summary</h2>
<p>Whether it&rsquo;s programming or other skills, I feel they all follow the &ldquo;Matthew Effect&rdquo; - the more I learn, the more I understand, and the faster you&rsquo;ll learn new things.</p>
<p>After writing a lot of code, I realized that a programmer&rsquo;s competitiveness isn&rsquo;t writing code, nor is it any particular language or framework.
The core competitiveness is the ability to solve problems through technology - why be constrained by any specific programming language or technology?</p>
<p>I hope ten years of programming is just a starting point, and in ten years I can write another piece: &ldquo;Reflections on Twenty Years of Programming.&rdquo;</p>
<div class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1">
<p><a href="https://struts.apache.org/">https://struts.apache.org/</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2">
<p><a href="https://spring.io/projects/spring-framework">https://spring.io/projects/spring-framework</a>&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3">
<p><a href="https://hibernate.org/">https://hibernate.org/</a>&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:4">
<p><a href="https://docs.spring.io/spring-framework/reference/web/webmvc.html">https://docs.spring.io/spring-framework/reference/web/webmvc.html</a>&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:5">
<p><a href="https://ibatis.apache.org/">https://ibatis.apache.org/</a>&#160;<a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:6">
<p><a href="https://mybatis.org/mybatis-3/">https://mybatis.org/mybatis-3/</a>&#160;<a href="#fnref:6" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:7">
<p><a href="https://github.com/sofastack/sofa-rpc">https://github.com/sofastack/sofa-rpc</a>&#160;<a href="#fnref:7" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:8">
<p><a href="https://spring.io/projects/spring-boot">https://spring.io/projects/spring-boot</a>&#160;<a href="#fnref:8" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:9">
<p><a href="https://github.com/sofastack/sofa-boot">https://github.com/sofastack/sofa-boot</a>&#160;<a href="#fnref:9" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</div>
]]></content:encoded>
    </item>
  </channel>
</rss>
