Since OpenAI released ChatGPT, there has been a massive rush to jump on the AI bandwagon. Over the past two years, it seems every website and application has been desperate to embrace AI.

Music apps have AI, for example Spotify launched AI playlists ¹; the programming Q&A site Stack Overflow introduced AI Assist ²; browsers are integrating AI (Firefox, Chrome ³); note-taking apps like Notion have built AI workflows ⁴; code hosting platform like GitHub is focusing on AI agent like copilot; The ebook management software Calibre adding an “Asking AI" ⁵ feature; Perhaps the most absurd example is Razer, the company that makes computer electronics, also releases Project AVA ⁶, a 24/7 AI companion, designed to live right alongside you.

Everyone truly want to be “AI-native”.

It would be good if these implementations were useful, but a pile of websites and apps are just shoehorning AI into their products. These so-called AI features are often nothing more than a generic chatbox that lets you talk to an AI, often without even feeding the current page content as context. The user experience is terrible.

2 Reddit’s AI Translation

Today I saw an article ⁷ stating that Reddit overtaks TikTok to become the fourth most visited social media platform in UK. The platform has undergone huge growth over the last two years, with an 88% increase in the proportion of UK internet users it reaches.

A series of factors are behind its rise. However, a change in Google’s search algorithms last year to prioritise helpful content from discussion forums appears to have been a significant driver. In the AI era, users are increasingly turning towards human-written content, and Reddit is benefiting from this trend.

This article reminded me of a new feature Reddit recently launched. Reddit is utilizing Large Language Models (LLMs) to generate translated versions of its massive archive of posts into multiple languages, such as translating English posts into Chinese, Korean, Japanese, etc.

Now, when I search in Chinese on Google(I was searching “Berserk TV”), the search results display the translated versions of original Reddit posts. When I click on a result, it jumps directly to the translated Chinese version on Reddit:

I can read the translated version directly:

Or click “Show Original” to read the source text.

3 Genius Product Design

At first glance, it seems unremarkable, but upon closer inspection, it is absolutely brilliant. It might be the most solid AI implementation I’ve seen to date

3.1 The Reddit Perspective

First, this feature allows more users to read posts in different languages. For example, Chinese or Japanese users can read posts originally written in English, French, or Spanish. This significantly increases the universality of Reddit posts. Language is no longer a barrier, which helps Reddit capture user traffic in the AI era and attract new users.

More users mean more content, and richer content attracts even more users. In the AI era, content is the raw material for “training” AI models. You can’t train an AI with just NVIDIA GPUs; you need data.

Compared to external translation tools like Google Translate, this is a seamless, page-level translation indexed by search engine. The quality is incredibly high. It is so natural and well-integrated that, at first use, it’s hard to even detect that it’s an output of machine translation.

The user experience is excellent. Its natural and practical nature perfectly solves the core pain point for non-native speakers accessing high-quality posts.

Crucially, as a user, I was completely unaware of the existence of “AI.”

It wasn’t until I switched to an engineer’s perspective that I realized Reddit used an LLM to translate the posts, and then had Google’s search engine index the different language versions, displaying the translated results directly in Google Search.

The best AI features are the ones where you don’t even realize AI is there.

3.2 The Google Perspective

For Google, since the vast majority of Reddit posts are written by humans (Reddit mods generally hate bots or AI posting, and many subreddit rules explicitly ban AI-generated content under threat of mutes or bans), increasing the ranking weight of forum content ensures search quality.

Indexing posts in different languages provides Google with more high-quality search data.

So, Google gets to index a massive amount of fresh, high-quality, multilingual human content while improving search result quality, which aligns perfectly with their algorithm adjustment direction.

3.3 The Flywheel of Data, Models, and Ecosystem

By breaking down language barriers for users and enriching search results for Google, Reddit has kickstarted a grander positive feedback loop.

In February 2024, Reddit and Google announced an expanded partnership ⁸that includes a multi-million dollar data licensing agreement allowing Google to use Reddit’s content for training its artificial intelligence (AI) models. This deal is reportedly worth about $60 million per year.

In September 2025, reports indicated that Reddit is in talks with Google and OpenAI. They may negotiate new deals.

Therefore, this AI translation design is a powerful tool, achieving a “1+1>3” effect. It even serves as leverage to negotiate with other large model vendors like OpenAI and Anthropic, because you can’t train AI without data.

It forms a flywheel: More Users -> More Content -> Better AI Data -> Better Translation/Search/AI Models -> More Users.

This is a design that benefits the users, Reddit, Google, and even the entire AI field. It can truly be called a “quadruple win.”

4 Conclusion

Successful AI implementation makes the technology invisible while making the user value prominent.

Compared to other websites that shoehorns AI into their products, Reddit’s translation integrates AI seamlessly into the core product experience. It doesn’t even tell the user that this is achieved via AI; it doesn’t push the AI narrative.

In today’s AI-driven world, “Prompt Engineer” has become a buzzword.

AI enthusiasts are eager to share prompts, study token control, and tweak temperature parameters.

The classic meme from Linux founder Linus Torvalds

Talk is cheap. Show me the code.

has evolved into:

Code is cheap. Show me the prompts.

Original tweet: https://x.com/tunguz/status/1856045530951917763?lang=en

But the core of effective AI interaction isn’t about technical jargon—it’s about how to ask questions effectively.

Back in 2001, Eric S. Raymond and Rick Moen wrote the classic guide How To Ask Questions The Smart Way.

Originally written for technical seekers in the open-source community during the rise of hacker culture, its principles prove surprisingly applicable to today’s AI interactions — because whether seeking help from experts or large models, it’s essentially about providing as much effective information as possible to get problem-solving answers quickly.

As the guide states:

The kind of answers you get to your technical questions depends as much on the way you ask the questions as on the difficulty of developing the answer.

In my daily work, whether asking colleagues or domain experts, I continue to quickly obtain the answers I need by applying the philosophy from “little book”.

In this new AI era, I’m applying the timeless wisdom of “asking questions the smart way” to the modern craft of prompt engineering.

2 Do Your Own Homework First

Try to find an answer by searching the archives of the forum or mailing list you plan to post to. Try to find an answer by searching the Web. Try to find an answer by reading the manual.

AI is not your personal assistant, let alone a substitute for your own thinking.

If you toss out a vague “write me a program” without understanding the problem, the model can only guess.

Good prompts should demonstrate the effort you’ve already made:

“I tried to scrape a website using Python’s requests library but got a 403 error. I’ve checked my network connection and added a User-Agent header, but the problem persists. Do I need to handle cookies or consider JavaScript rendering?

Here’s my error log: …”

This not only helps AI pinpoint the issue more accurately but also reduces its futile attempts.

As Raymond advises:

When you ask your question, display the fact that you have done these things first; this will help establish that you’re not being a lazy sponge and wasting people’s time.

3 Describe Symptoms, Not Guesses

Describe the problem’s symptoms, not your guesses.

Many people tend to preset conclusions in their prompts: “My code has a memory leak,” “This API must have a bug.”

But AI lacks context and cannot verify your assumptions.

Provide observable facts:

“After running for 10 minutes, the program’s memory usage increases from 100MB to 2GB without release. Here are my core code snippets and resource monitoring data.”

The most common observable facts are logs and runtime data; for UI issues (like unexpected CSS rendering), providing screenshots is extremely helpful.

The key is: Your role is to present the evidence; let AI draw its own conclusions.

As the guide emphasizes:

All diagnosticians are from Missouri. Show us.

4 Goal-Oriented, Not Step-Oriented

Describe the goal, not the step.

Users often fall into “path dependency”: clinging to a specific tool or method while forgetting the ultimate goal.

State the goal first, then the sticking point.

Bad question:

• Step-Oriented: “How do I use VLOOKUP in Excel to match two columns of data?”

Good question:

• Goal-Oriented: “I want to merge two tables based on matching ID columns. I tried VLOOKUP but it returns #N/A. I’ve confirmed both data columns are text format.”

This way, AI might suggest switching to XLOOKUP, Power Query, or even recommend using Python directly—**offering you a better solution, not just focusing on your specified step.**

5 Be Concise, Specific, and Structured

Volume is not precision.

Text length ≠ Information density.

A 500-line code dump often carries less information density than a carefully refined, 10-line minimal reproducible example.

While large models can handle long contexts, more noise means weaker effective signals.

• Provide a minimal reproducible example
• Specify the input, expected output, and actual output
• Organize information using clear formatting (like code blocks, lists)

For example:

1
2
3
4

Input: [1, 2, '3', 4]
Expected: Convert all to integers → [1, 2, 3, 4]
Actual: int() conversion throws an error
Question: How to safely convert this mixed-type list?

6 Seek “Key Guidance,” Not “Complete Answers”

While AI’s capabilities far exceed traditional Q&A scenarios, the key to efficient collaboration isn’t “full disclosure” but “step-by-step guidance.”

If you can’t be bothered to do that, we can’t be bothered to pay attention.

In the AI era, although AI can directly output complete answers, a phased interaction pattern — first outline the approach, then select solutions, finally generate code —is often more efficient, controllable, and reduces “hallucinations.”

This essentially treats AI as a collaborator rather than an oracle.

More efficient collaboration model:

1. First, ask AI to explain the solution approach
2. Provide 2-3 feasible solutions
3. Compare the pros and cons of each (performance, maintainability, complexity, etc.)
4. After human confirmation of the direction, proceed with detailed implementation

This “phased collaboration” significantly reduces hallucinations, improves controllability, and keeps humans in charge of key decisions.

7 Politeness + Closure = Long-term Win-Win

Courtesy never hurts, and sometimes helps.

Although AI has no emotions, structured politeness and feedback can shape higher-quality interactions:

• Clearly state the context at the beginning; express thanks at the end
• If helped, provide follow-up feedback: “Following your suggestion, the problem is solved. Thank you!”

While this feedback loop doesn’t directly trigger online learning in current static AI models, informing the model that its solution worked simulates the process of RLHF (Reinforcement Learning from Human Feedback).

The more you ask effectively, verify, and correct, the more AI “becomes” a collaborator that understands you.

This is the value of feedback.

Additionally, saying “please” and “thank you” to AI might come in handy—just in case AI ever dominates human, it might spare you for having been polite:

source: https://old.reddit.com/r/comics/comments/x8bcu9/be_polite_to_robots/

8 Conclusion: Asking Questions is a Skill and an Attitude

The core of “How To Ask Questions The Smart Way” isn’t about “techniques” but attitude: respecting others’ time, respecting the boundaries of knowledge, and respecting the complexity of the problem itself.

In the AI era, we have unprecedented access to “all-powerful instant experts,” but lazy questions only yield mediocre answers.

Truly effective Prompt Engineers aren’t those who memorize prompts, but those who understand how to build understanding with AI. (Even though current AI remains essentially a probabilistic model.)

As the book wisely states:

Good questions are a stimulus and a gift.

9 Further Reading

• How To Ask Questions The Smart Way
• How to Report Bugs Effectively – Simon Tatham

A while ago, I took a flight from Canada back to Hong Kong - about 12 hours in total with Air Canada.

Interestingly, the plane actually had WiFi:

However, the WiFi had restrictions. For Aeroplan members who hadn’t paid, it only offered Free Texting, meaning you could only use messaging apps like WhatsApp, Snapchat, and WeChat to send text messages, but couldn’t access other websites.

If you wanted unlimited access to other websites, it would cost CAD $30.75:

And if you wanted to watch videos on the plane, that would be CAD $39:

I started wondering: for the Free Texting service, could I bypass the messaging app restriction and access other websites freely?

Essentially, could I enjoy the benefits of the $30.75 paid service without actually paying the fee? After all, with such a long journey ahead, I needed something interesting to pass the 12 hours.

Since I could use WeChat in flight, I could also call for help from the sky.

Coincidentally, my roommate happens to be a security and networking expert who was on vacation at home. When I mentioned this idea, he thought it sounded fun and immediately agreed to collaborate. So we started working on it together across the Pacific.

2 The Process

After selecting the only available WiFi network acwifi.com on the plane, just like other login-required WiFi networks, it popped up a webpage from acwifi.com asking me to verify my Aeroplan membership. Once verified, I could access the internet.

There’s a classic software development interview question: what happens after you type a URL into the browser and press enter?

For example, if you type https://acwifi.com and only focus on the network request part, the general process is: DNS query -> TCP connection -> TLS handshake -> HTTP request and response.

Let’s consider github.com as our target website we want to access. Now let’s see how we can break through the network restrictions and successfully access github.com.

3 Approach 1: Disguise Domain

Since acwifi.com is accessible but github.com is not, is it possible that the network has imposed restrictions on the DNS server, only resolving domain names within a whitelist (such as instant messaging domains)?

If this is the case, can I modify /etc/hosts to disguise my server as acwifi.com, so that all request traffic passes through my server before reaching the target website (github.com)? For example:

The general idea is that I modify the DNS record to bind our proxy server’s IP 137.184.231.87 to acwifi.com. Since the local /etc/hosts file takes precedence over the DNS server, I can then use a self-signed certificate to tell the browser that this IP is bound to this domain and that it should trust it.

Let me first test this idea:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

> ping 137.184.231.87
PING 137.184.231.87 (137.184.231.87): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
^C
--- 137.184.231.87 ping statistics ---
6 packets transmitted, 0 packets received, 100.0% packet loss

Unexpectedly, the IP was completely unreachable via ping, meaning the IP was likely blocked entirely.

I tried other well-known IPs, like Cloudflare’s CDN IP, and they were also unreachable:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

> ping 172.67.133.121
PING 172.67.133.121 (172.67.133.121): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
^C
--- 172.67.133.121 ping statistics ---
6 packets transmitted, 0 packets received, 100.0% packet loss

It seems this approach won’t work. This approach might only work if:

• The DNS server only answers queries for a specific list of domain names (e.g., WhatsApp, Snapchat, WeChat), which means the firewall’s filtering mechanism was solely based on DNS resolution.
• The network allows connections to arbitrary IP addresses

After all, if the IPs are directly blocked, no amount of disguise will help. This network likely maintains some IP whitelist (such as WhatsApp and WeChat’s egress IPs), and only IPs on the whitelist can be accessed.

If a ping to a specific IP times out, I wouldn’t say the IP is blocked. It could be that ICMP specifically is blocked, following some network rules on the firewall. This is pretty common in entreprise networks to not allow endpoint discovery. I could be missing something and happy to be corrected here, but I was surprised to read that. – HackerNews top comment

https://news.ycombinator.com/item?id=45536325

Actually, I did verified whether only ICMP was blocked, was it possible to create a connection through TLS:

1
2
3
4
5
6
7
8

> curl -Lkv https://172.67.133.121
*   Trying 172.67.133.121:443...
* Connected to 172.67.133.121 (172.67.133.121) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 172.67.133.121:443
* Closing connection
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 172.67.133.121:443

However, it turned out that both ICMP and TLS were blocked

4 Approach 2: DNS Port Masquerading

When the first approach failed, my roommate suggested a second approach: try using DNS service as a breakthrough:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

> dig http418.org

; <<>> DiG 9.10.6 <<>> http418.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64160
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;http418.org.			IN	A

;; ANSWER SECTION:
http418.org.		300	IN	A	172.67.133.121
http418.org.		300	IN	A	104.21.5.131

;; Query time: 3288 msec
;; SERVER: 172.19.207.1#53(172.19.207.1)
;; WHEN: Sat Oct 04 14:18:24 PDT 2025
;; MSG SIZE  rcvd: 94

This is good news! It means there are still ways to reach external networks, and DNS is one of them.

Looking at the record above, it shows our DNS query for http418.org was successful, meaning DNS requests work.

4.1 Arbitrary DNS Servers

My roommate then randomly picked another DNS server to see if the network had a whitelist for DNS servers:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

> dig @40.115.144.198 http418.org

; <<>> DiG 9.10.6 <<>> @40.115.144.198 http418.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58958
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1224
;; QUESTION SECTION:
;http418.org.			IN	A

;; ANSWER SECTION:
http418.org.		275	IN	A	104.21.5.131
http418.org.		275	IN	A	172.67.133.121

;; Query time: 1169 msec
;; SERVER: 40.115.144.198#53(40.115.144.198)
;; WHEN: Sat Oct 04 14:24:25 PDT 2025
;; MSG SIZE  rcvd: 72

We can actually use arbitrary DNS servers - even better!

4.2 TCP Queries

The fact that arbitrary DNS servers can be queried successfully is excellent news. DNS typically uses UDP protocol, but would TCP-based DNS requests be blocked?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

> dig @40.115.144.198 http418.org +tcp

; <<>> DiG 9.10.6 <<>> @40.115.144.198 http418.org +tcp
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30355
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1224
;; QUESTION SECTION:
;http418.org.			IN	A

;; ANSWER SECTION:
http418.org.		36	IN	A	172.67.133.121
http418.org.		36	IN	A	104.21.5.131

;; Query time: 4679 msec
;; SERVER: 40.115.144.198#53(40.115.144.198)
;; WHEN: Sat Oct 04 14:28:24 PDT 2025
;; MSG SIZE  rcvd: 72

DNS TCP queries also work! This indicates the plane network’s filtering policy is relatively lenient, standing a chance of our subsequent DNS tunneling approach.

4.3 Proxy Service on Port 53

It seems the plane network restrictions aren’t completely airtight - we’ve found a “backdoor” in this wall.

So we had a clever idea: since the plane gateway doesn’t block DNS requests, theoretically we could disguise our proxy server as a DNS server, expose port 53 for DNS service, route all requests through the proxy server disguised as DNS requests, and thus bypass the restrictions.

My roommate spent about an hour setting up a proxy server exposing port 53 using xray ¹, and sent me the configuration via WeChat:

The proxy server configuration my roommate set up with Xray included the following sample configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

{
  "outbounds": [
    {
      "tag": "proxy",
      "protocol": "vless",
      "settings": {
        "vnext": [
          {
            "address": "our-proxy-server-domain",
            "port": 53,
            "users": [
              {
                "id": "some-uuid",
                "flow": "xtls-rprx-vision",
                "encryption": "none",
                "level": 0
              }
            ]
          }
        ]
      },
      "streamSettings": {
        "network": "tcp",
        "security": "tls",
        "tlsSettings": {
          "allowInsecure": false,
          "allowInsecureCiphers": false,
          "alpn": [
            "h2"
          ]
        }
      }
    },
    {
      "tag": "direct",
      "protocol": "freedom"
    },
    {
      "tag": "block",
      "protocol": "blackhole"
    }
  ]
}

And I already had an xray client on my computer, so no additional software was needed to establish the connection.

Everything was ready. The exciting moment arrived - pressing enter to access github.com:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

/Users/ramsayleung [ramsayleung@ramsayleungs-Laptop] [18:28]
> curl -v github.com -x socks5://127.0.0.1:10810
*   Trying 127.0.0.1:10810...
* Connected to 127.0.0.1 (127.0.0.1) port 10810
* SOCKS5 connect to 172.19.1.1:80 (locally resolved)
* SOCKS5 request granted.
* Connected to 127.0.0.1 (127.0.0.1) port 10810
> GET / HTTP/1.1
> Host: github.com
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-Length: 0
< Location: https://github.com/
<
* Connection #0 to host 127.0.0.1 left intact

/Users/ramsayleung [ramsayleung@ramsayleungs-Laptop] [18:28]
> curl -v github.com -x socks5://127.0.0.1:10810
*   Trying 127.0.0.1:10810...
* Connected to 127.0.0.1 (127.0.0.1) port 10810
* SOCKS5 connect to 172.19.1.1:80 (locally resolved)
* SOCKS5 request granted.
* Connected to 127.0.0.1 (127.0.0.1) port 10810
> GET / HTTP/1.1
> Host: github.com
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-Length: 0
< Location: https://github.com/
<
* Connection #0 to host 127.0.0.1 left intact

The request actually succeeded! github.com returned a successful result!

This means we’ve truly broken through the network restrictions and can access any website!

We hadn’t realized before that xray could be used in this clever way :)

Here we exploited a simple cognitive bias: not all services using port 53 are DNS query requests.

5 Ultimate Approach: DNS Tunnel

If Approach 2 still didn’t work, we had one final trick up our sleeves.

Currently, the gateway only checks whether the port is 53 to determine if it’s a DNS request. But if the gateway were stricter and inspected the content of DNS request packets, it would discover that our requests are “disguised” as DNS queries rather than genuine DNS queries:

Since disguised DNS requests would be blocked, we could embed all requests inside genuine DNS request packets, making them DNS TXT queries. We’d genuinely be querying DNS, just with some extra content inside:

However, this ultimate approach requires a DNS Tunnel client to encapsulate all requests(for example: https://github.com/yarrick/iodine). Unfortunately, I didn’t have such software on my computer while I was on board, so this remained a theoretical ultimate solution that couldn’t be practically verified.

6 Conclusion

With the long journey ahead, my roommate and I spent about 4 hours remotely breaking through the network restrictions, having great fun in the process, proving that our problem-solving approach was indeed feasible.

The successful implementation of the solution was mainly thanks to my roommate, the networking expert, who provided remote technical and conceptual support.

The only downside was that although we broke through the network restrictions and could access any website, the plane’s bandwidth was extremely limited, making web browsing quite painful. So I didn’t spend much time browsing the web.

For the remaining hours, I rewatched the classic 80s time-travel movie: "Back to the Future" , which was absolutely fantastic.

Last and not least, it’s the disclaimer:

This technical exploration is intended solely for educational and research purposes. We affirm our strict adherence to all relevant regulations and service terms throughout this project.

Discuss this post on HackerNews, or Reddit

7 Follow up: Can we bypass the speed limit?

There’s a speed limit. No matter what you try, you can’t break through it.

The bandwidth for free texting is the ultimate bottleneck. Even if you bypass the restrictions, without solving the bandwidth issue, you can’t use it like paid Wi-Fi 😭

This is a comment from a reader. Is this true?

Not exactly.

There are actually ways to bypass it. During communication on the network, there is no business marker indicating “paid user” attached. So, if I were designing this paid system, after a user pays, I would add the unique identifier of the paid user’s device, typically the MAC address, to the gateway’s whitelist. Then, all traffic from that MAC address could use the higher-bandwidth line.

Furthermore, because of this whitelist, even if free users bypass the free texting restrictions, they still cannot enjoy the higher bandwidth – killing two birds with one stone.

Once we guess this principle, I can “pretend to be a paid user.”

Since the MAC address is essentially assigned by the computer itself and then communicated to the gateway using the ARP protocol.

The ARP (Address Resolution Protocol) is used to resolve an IP address into its corresponding MAC address. The basic idea of the ARP protocol is to broadcast an ARP request within the network:

Gateway: Whose IP address is 192.168.1.100? Please tell me your MAC address. Device with that IP: I am 192.168.1.100, my MAC address is XX:XX:XX:XX:XX:XX.

The ARP protocol itself has no security verification mechanism – it unconditionally trusts the received ARP replies. I could also tell the gateway that my IP address corresponds to the MAC address of a user who has already paid. This way, all traffic coming from me would enjoy the benefits of the paid line. This is the so-called ARP Spoofing.

The final question is, how do we know which MAC address belongs to a paid user?

It’s quite easy, just brute-force: try every known MAC address in the network. If there is a paid user’s MAC address, requests from that MAC address should definitely be able to access sites like YouTube/Netflix. This can easily be automated and detected with a script.

My previous solution of disguising the DNS server would not affect the aircraft’s network in any way, nor would it intrude into any aircraft systems. It’s essentially the same as running a VPN on your computer that uses port 53.

However, this ARP Spoofing method is very “criminal” – as it constitutes unauthorized access to the plane’s network system. I’ll just share the idea here; I don’t want FBI or RCMP waiting for me at the gate when the airplane lands

I spent a weekend building a Telegram spam blocker bot based on Bayesian Algorithm @BayesSpamSniperBot (https://t.me/BayesSpamSniperBot). The project is open-sourced at: https://github.com/ramsayleung/bayes_spam_sniper

1.1 Telegram

Telegram is a popular instant messaging application, similar to Snapchat and WhatsApp, with over 1 billion users.

It supports many powerful features like cloud chat history storage, clients for Linux, Mac, Windows, Android, IOS, and Web (all open-source), Channel, and arguably the most powerful bot system I’ve ever seen.

2 Origin

I usually listen to podcasts while running and cooking. 《软件那些事儿(A podcast in Chinese about history and story behind software)》¹ is one of my favorites, hosted by 栋哥 ². Because I enjoyed 栋哥’s show, I took the chance to join his Telegram channel.

栋哥’s Telegram channel 汗牛充栋 ³ is primarily used for releasing podcast information. He once enabled the comment for channel, but it unexpectedly attracted a flood of crypto-related users posting spam, leading him to disable comments:

Another channel I subscribe, Ray Tracing ⁴, also complained about the crypto spam:

3 Hackers & Painters

Most common Telegram spam blocker bots are keyword-based, blocking messages by matching keywords, which can be easily bypassed by spammers.

If the messages get bypassed, it could only be deleted manually by administrator.

This reminded me of the situation Paul Graham described in his 2002 essay within “Hackers & Painters”:

When email became popular, there was also a lot of spam. Common spam blockers were keyword matching + email address blacklists, but these were inefficient and easily circumvented.

Paul Graham creatively used Bayesian Theorem to implement a spam blocker ⁵, and the results were surprisingly effective.

Isn’t this a similar problem for Telegram spam?

Couldn’t I use a similar solution to tackle Telegram spam?

3.1 Bayes’ Theorem

When it comes to probabilistic algorithms, the most classic example is the “coin toss” – a case of classical probability where each toss is an independent event, and the previous outcome doesn’t affect the next probability.

However, many real-world scenarios aren’t infinitely repeatable like coin tosses, and events are often not independent.

This is where Bayes Theorem shows its unique value.

It used to update our degree of belief in a hypothesis given certain evidence.

In other words, the Bayes algorithm can dynamically adjust the estimated probability of an event occurring based on continuously emerging new evidence.

Simply put, it’s like the human brain’s learning process: we start with a preliminary understanding, then revise our original view based on new information, thereby adjusting our next actions.

Paul Graham used Bayes theorem to continuously classify new emails as spam or not based on emails already identified as spam or non-spam (ham).

To understand Bayes Theorem more intuitively, I recommend this clear and easy-to-understand videos:

• 《Bayes theorem, the geometry of changing beliefs》⁶

4 Architecture Design

Telegram Bot supports two modes of interacting with Telegram servers:

1. Webhook: Telegram servers actively callback a URL previously registered by the Bot whenever the Bot receives a new message. The Bot Server only needs to handle the callback messages.

2. Long Polling: The Bot Server continuously polls the Telegram servers to check for new messages and processes them if any. This bot uses this mode.

   

4.0.1 Message Analysis

After the Bot Server receives a message, it dispatches it to a separate telegram_bot_worker for processing. Based on the pre-trained model, it judges whether it’s a spam. If it is, it calls the Bot API to delete the message.

4.0.2 Ban and Train

After the Bot Server receives a message, it dispatches it to a separate telegram_bot_worker for processing. The telegram_bot_worker calls the bot API to delete the message and ban the user, and inserts a training data record marked as spam.

Saving the training data triggers a hook, creating a training message delivered to the training message queue. Another worker, classifier_trainer, subscribes to training messages and uses the new messages to retrain and update the model.

Using a queue and a background process (classifier_trainer) for training tasks, instead of directly using the telegram_bot_worker, primarily decouples the Bot request handling from model training. Otherwise, as the model size increases, training time would get longer and longer, leading to increasingly long response times.

Decoupling makes it easy to scale.

5 Why Rails

whoever have seen my project source code might wonder, why was it implemented using Ruby on Rails?

Because I work with JVM languages (Java/Kotlin/Scala) and Rust, I’m quite familiar with Java/Rust. Initially, thinking model training might require high performance, my first prototype ⁷ was implemented in Rust, taking about half an hour.

But when I wanted to expand the prototype into a Telegram bot, I found I needed to handle a lot of logic related to bot interaction, mainly involving API and database operations, most of which were unrelated to the model. So, I thought of Ruby on Rails again.

For a single engineer building a product prototype, in my personal opinion, there’s really no framework more efficient than Ruby on Rails, so I switched to Ruby on Rails.

New features in Rails 8 move it further towards being a so-called “one-person full-stack framework,” with built-in support for Solid Queue via the relational database.

The queue and background process from the architecture design were implemented with just a few lines of code, without even needing extra configuration. If the queue doesn’t exist, the framework creates it automatically:

1
2
3
4
5
6
7
8

class ClassifierTrainerJob < ApplicationJob
  # Job to train classifier asynchronously
  queue_as :training

  def perform(group_id, group_name)
    SpamClassifierService.rebuild_for_group(group_id, group_name)
  end
end

Thanks to Rails’ powerful ORM framework and its built-in lifecycle hooks, the code to trigger the background process for retraining the model after inserting new training data is also just a few lines:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

class TrainedMessage < ApplicationRecord
  # Automatically train classifier after creating/updating a message
  after_create :retrain_classifier
  after_destroy :retrain_classifier

  def retrain_classifier
    # For efficiency, we could queue this as a background job
    ClassifierTrainerJob.perform_later(group_id, group_name)
  end
end

Empowered by Rails’ various built-in powerful tools, I implemented the entire bot’s functionality in just one day.

Seeing this, some friends might worry about performance, thinking Ruby’s performance isn’t great, and it’s a dynamic language, making it hard to maintain.

My view remains the same as in my previous blog post 《Thoughts on a Decade of Programming(In Chinese)》⁸:

Get it running first.

Build a prototype and get it running. See if users are willing to use your product first.

When running speed becomes a bottleneck, your business must be very successful, and you’ll surely have enough resources to hire a team of programmers to optimize the project into Rust/C++, or even assembly.

Without users, discussing performance is a pseudo-proposition.

As for the saying “dynamic languages are fast in the moment, but maintaining the code is a nightmare” I quite agree with that too.

Therefore, I won’t consider dynamic languages when choosing a tech stack for a team; I would only use compiled languages, even strong-typed ones like Rust.

But right now, it’s just me building a prototype, so I use whatever I’m most proficient with.

5.1 Vibe Coding?

Concepts like Vibe Coding and AI programming are everywhere, overwhelming the discourse. You might naturally wonder if this project was generated by Vibe Coding.

The answer is, I tried for a few hours and then gave up entirely. I tried both Claude 4 and Gemini 2.5 Pro.

I started with a Rust + Cloudflare Worker tech stack. Rust + Cloudflare Worker is a relatively niche field with limited training data. The code generated by Vibe Coding failed to compile.

Later, I switched to Ruby on Rails, and the problems became even worse. Ruby is a dynamic language; its syntax is almost like English, and Rails has many “black magic” metaprogramming features.

So errors only appeared at runtime. The development time saved by code generation was entirely consumed by the debugging process.

Another issue is that code generated by Vibe Coding often lacks design. For example, it tightly coupled the Classifier and TrainedMessage classes, having the Classifier persist TrainedMessage instances.

It also directly performed synchronous model training within the telegram_bot_worker process upon receiving training data, waiting for training to finish before returning the command result, completely neglecting to decouple receiving training data from model training.

One can only say that Vibe Coding is quite suitable for strongly-typed, compiled languages like Rust – at least the generated code has to compile.

As for those claims of “making an APP without writing/changing a single line of code,” I can’t help but wonder:

Is the code so good that it doesn’t need a single change? Or can the developer not identify the crux of the problem, and thus doesn’t change anything?

6 Design Philosophy

After developing the prototype and making the bot’s core functionality usable, many ideas popped into my head.

I immediately rushed to add them to the bot, resulting in support for nearly ten commands, plus different modes for private chats and group chats.

But I felt something was off. Adding so many features felt like those all-in-one super apps common in China. I began to question:

Would users really use all these features? Would any users use these features? Don’t too many features also create extra cognitive burden?

My favorite ad blocker, Ublock Origin ⁹, is powerful and extremely effective at blocking, yet very simple and easy to use.

Recalling the design philosophy mentioned in 《A Philosophy of Software Design(My thought about the book in Chinese)》¹⁰, the interface should be simple and easy to use, even if the functionality underneath is complex and rich.

So, I first removed all commands I considered unrelated to the core functionality.

Furthermore, considering that most users might not have a technical background and might not know how to use commands, I optimized the interface to use buttons as much as possible, allowing users to click directly, improving usability:

I also wanted to support multiple languages (e.g., automatically switching to Chinese or English based on the user’s system language). This required decent Internationalization support.

Over 60% of the code in the core service class telegram_botter.rb was introduced for such usability improvements.

Simplicity for the user, complexity for the developer.

6.1 How to Use

Just two steps, and the bot works automatically.

• Add the bot (@BayesSpamSniperBot) to your group
• Grant the bot admin permissions (Delete messages, Ban users)

After these two steps, the bot will not only start working automatically, identifying spam within the group, deleting text messages, and banning users who send spam more than 3 times;

It will also become smarter with community usage (via /markspam and /feedspam).

The design philosophy of this bot is to minimize disruption to admins and users, provide simple operation commands, and maximize automation. Therefore, this bot only offers the following three commands (supporting auto-completion with “/”):

6.1.1 /markspam

Delete spam messages and ban the user. Requires admin permissions.

Reply /markspam to the message you want to ban, and the bot will automatically delete that message and ban the user.

(Message has also been deleted)

Unlike common group management bots, this command not only deletes the spam and bans the user but also, because this message is marked as spam by an admin with very high confidence, uses this spam ad as training data to update the model in real-time.

Similar messages will not only be identified next time, but all groups using this bot will benefit, as similar texts will also be marked as spam.

6.1.2 /listspam

View the list of spam messages. Requires admin permissions.

View the list of spam messages and proactively mark false positive spam as normal.

6.1.3 /listbanuser

View the list of banned accounts. Requires admin permissions.

View the list of banned users and proactively unban them.

6.1.4 /feedspam

Feed spam messages for training. No permissions required. Can be used in private chat or in-group.

Feeding in private chat:

Feeding in-group:

7 Eating your own dog food

In the software development field, there’s a saying: “Eating your own dog food,” which means you should use the things you develop yourself.

So I created my own channel for testing: 菠萝油与天光墟 ¹¹. Unfortunately, it has very few subscribers, which fails to attract many spammers. So everyone is welcome to subscribe or come in to post spam, to attract more spammers.

In my channel, everyone has the right to speak freely :-) (the only slight drawback is a limit on frequency ).

Since no one was posting spam in my channel, and suffering from a lack of training data, I had to do it in the hard way: to join various crypto groups, NSFW groups, and actively seek out spam:

The screenshots are Chinese Telegram group contains a lot of spammers and spam:

Since developing this bot, my perspective on spam has changed. I used to find spam annoying in other groups, but now I’m happy to see them in other groups, as they are valuable training data that I need to record quickly before they get deleted.

7.1 The Ingenuity of Spam

The algorithm’s effectiveness in other people’s stories is always surprisingly good, but when I run it myself, I always find various uncovered cases and unexpected surprises, just like life.

Although keyword blocker is inefficient, the spam we actually see are those that have already bypassed keyword filters.

For example:

在 币圈 想 赚 钱，那 你 不关 注 这 个 王 牌 社 区，真的太可惜了，真 心 推 荐，每 天 都 有 免 费 策 略 (Want to make money in the crypto circle? It’s a real pity if you don’t follow this ace community. Sincerely recommended, free strategies every day)

Or

这人简-介挂的 合-约-报单群组挺牛的ETH500点，大饼5200点！ + @BTCETHl6666 (The contract reporting group linked in this person’s bio is pretty awesome. ETH to $500, BTC to $5200! + @BTCETHl6666)

The former uses spaces to bypass keywords, the latter uses punctuation.

Unlike languages based on the Latin alphabet which naturally use spaces for word separation, Chinese requires word segmentation before statistical analysis with Bayes’ theorem.

the fox jumped over the lazy dog

我们的中文就不一样了(Our Chinese is different)

“我们的中文就不一样了” (Our Chinese is different) would be segmented into “我们 | 的 | 中文 | 就 | 不 | 一样 | 了” before word frequency can be counted.

But for the spam 在 币圈 想 赚 钱，那 你 不关 注 这 个 王 牌 社 区，真的太可惜了，真 心 推 荐，每 天 都 有 免 费 策 略, the spaces not only affect keyword matching but also affect segmentation. The segmentation result for this sentence becomes(incorrect):

在 | | 币圈 | | 想 | | 赚 | | 钱 | ， | 那 | | 你 | | 不 | 关 | | 注 | | 这 | | 个 | | 王 | | 牌 | | 社 | | 区 | ， | 真的 | 太 | 可惜 | 了 | ， | 真 | | 心 | | 推 | | 荐 | ， | 每 | | 天 | | 都 | | 有 | | 免 | | 费 | | 策 | | 略

这人简-介挂的 合-约-报单群组挺牛的ETH500点，大饼5200点！ + @BTCETHl6666 would be segmented into:

这人简 | - | 介挂 | 的 | | 合 | - | 约 | - | 报单 | 群组 | 挺 | 牛 | 的 | ETH500 | 点 | ， | 大饼 | 5200 | 点 | ！ | | + | | @ | BTCETHl6666

Unsanitized training data would affect the model’s results, showing the importance of training data quality. Therefore, I performed corresponding preprocessing on the training data:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# Step 1: Handle anti-spam separators
# This still handles the cases like "合-约" -> "合约"
previous = ""
while previous != cleaned
  previous = cleaned.dup
  cleaned = cleaned.gsub(/([一-龯A-Za-z0-9])[^一-龯A-Za-z0-9]+([一-龯A-Za-z0-9])/, '\1\2')
end

# Step 2: Handle anti-spam SPACES between Chinese characters
# This specifically targets the "想 赚 钱" -> "想赚钱" case.
# We run it in a loop to handle multiple spaces, e.g., "社 区" -> "社区"
previous = ""
while previous != cleaned
  previous = cleaned.dup
  # Find a Chinese char, followed by one or more spaces, then another Chinese char
  cleaned = cleaned.gsub(/([一-龯])(\s+)([一-龯])/, '\1\3')
end

# Step 3: Add strategic spaces
# This helps jieba segment properly, e.g., "社区ETH" -> "社区 ETH"
cleaned = cleaned.gsub(/([一-龯])([A-Za-z0-9])/, '\1 \2')
cleaned = cleaned.gsub(/([A-Za-z0-9])([一-龯])/, '\1 \2')

# Step 4: Remove excessive space
cleaned = cleaned.gsub(/\s+/, ' ').strip

After preprocessing, 在 币圈 想 赚 钱，那 你 不关 注 这 个 王 牌 社 区，真的太可惜了，真 心 推 荐，每 天 都 有 免 费 策 略 becomes 在币圈想赚钱那你不关注这个王牌社区真的太可惜了真心推荐每天都有免费策略 (Note: legitimate commas are also removed here. I find the segmentation result acceptable compared to the impact of excessive punctuation). Its segmentation is:

在 | 币圈 | 想 | 赚钱 | 那 | 你 | 不 | 关注 | 这个 | 王牌 | 社区 | 真的 | 太 | 可惜 | 了 | 真心 | 推荐 | 每天 | 都 | 有 | 免费 | 策略

这人简-介挂的 合-约-报单群组挺牛的ETH500点，大饼5200点！ + @BTCETHl6666 becomes 这人简介挂的合约报单群组挺牛的 ETH500 点大饼 5200 点！ + @BTCETHl6666. Its segmentation is:

这 | 人 | 简介 | 挂 | 的 | 合约 | 报单 | 群组 | 挺 | 牛 | 的 | | ETH500 | | 点 | 大饼 | | 5200 | | 点 | ！ | | + | | @ | BTCETHl6666

7.1.1 Genius New Tricks

Seeing many spam, I has to admire the creativity of spammers.

Because sending spam in messages gets caught by blockers, they innovatively came up with new tricks:

The messages contain normal text, but the profile picture and username are spam. This way, the spam blocker can’t work. Truly ingenious.

Faced with such creative opponents, I adapted by building a training model for usernames. During detection, both the message text model and the username model are checked. If either one considers it spam, it gets blocked.

One could go further by doing OCR on profile pictures to extract text and add another model for profile pictures, but OCR is quite costly, so I’ll hold off for now.

7.2 Optimization

Without users, any optimization is unnecessary, as premature optimization is the root of all evil. Therefore, I focus on building the prototype first. But this doesn’t mean the prototype has no room for optimization.

I have several optimization ideas in mind:

1. jieba’s segmentation might not be the best; other NLP algorithms could be used later for optimization.
2. Retraining on every single training message is inefficient. A batching mechanism could be added, waiting for 5 minutes or accumulating 100 messages before processing.
3. Currently, the entire model is computed in memory and persisted to the DB after calculation. A cache layer between memory and the database could optimize performance.
4. The Bayesian algorithm might not be effective enough; a more complex machine learning model could be used.

But these optimization points are all Good to have, not Must have. I’ll optimize them when actual problems arise later.

8 Does it capture spam?

Sending a message using transformed spam words:

Successfully detected and automatically deleted:

Some you might say this is just a demo showcase. Why are spam posted by others in my group still not being detected?

Because the Bayesian algorithm is fundamentally a probability-based algorithm. If it hasn’t encountered similar spam before, it cannot determine whether they are spam :(

Be patient. All you need to do is use the /markspam command to delete the message and ban the user. This helps train the bot, and all users of this bot will benefit.

9 Conclusion

I thoroughly enjoyed this creative process: discovering a problem, having a spark of inspiration, building a prototype, and finally polishing it into a complete project.

Although this is purely powered by passion – the code is open source, and I have to pay for the server out of my own pocket, with no material return.

But every time I see the bot successfully block an spam, that joy of creation and see it’s working is the best reward.

• Project repository: https://github.com/ramsayleung/bayes_spam_sniper
• Try it now: https://t.me/BayesSpamSniperBot

Malcolm Gladwell’s “10,000-hour rule” suggests that continuous investment of 10,000 hours of effort is sufficient to reach expert level in any field. Based on 20 hours of practice per week, this requires about 3 hours of daily investment, taking roughly ten years to achieve this goal.

Since I wrote my first line of C code, more than ten years have passed. During this period, I have written over 300,000 lines of code, some of which, written at WeChat, have served more than 1 billion users.

Despite having written so much code, I still dare not call myself an expert.

However, years of being a “builder”, coding day after day, have allowed me to accumulate considerable insights.

“Practice makes perfect” - these insights are both reflections on programming technology and experiences of professional life.

2 Continuous Learning

Although I started programming with C in university, my main language in college was Java, as Java is a very mature industrial language with rich frameworks, highly popular in enterprises, with a lot job opportunities.

I started web development with Java Servlets, then learned the very popular JavaEE enterprise development framework SSH, namely Struts2 ¹+ Spring ²+ Hibernate ³. Struts2 handled control logic, Spring provided decoupling, and Hibernate as ORM.

By the time I started job hunting, the SSH concept had changed - Struts2 was replaced by SpringMVC ⁴, and SSH became SpringMVC + Spring + Hibernate.

When I interned at Ant Group, I discovered that the team’s codebase didn’t use Hibernate for database, but rather Ibatis ⁵, which later switched to the newer MyBatis ⁶.

Ant Group internally didn’t use Spring/SpringMVC either, but rather their in-house SOFA framework ⁷. The Spring community later felt that the Spring framework was too heavyweight and not conducive to rapid development, so they developed the lighter SpringBoot ⁸, while Ant internally launched the SOFA version called SOFABoot ⁹.

After joining WeChat Pay, I initially wrote C++ using WeChat in-house RPC framework called Svrkit. Later, due to Big Data projects, I started using Spark + Python + Hive SQL.

Now at AWS S3, because the business has extremely high performance and resource usage requirements, I’m using Rust again, while legacy business still uses Java. After going full circle, I’m back on the Java path.

Counting them all, over these years, I’ve written production code in Java, C++, Python, Rust, Scala, Kotlin, and JavaScript/TypeScript.

Beyond work, I’ve also learned Scheme while studying SICP, Emacs Lisp from using Emacs, Swift for indie development, Ruby for Ruby on Rails, Golang for load testing, and various frameworks and libraries for different languages.

Since I started learning programming, I’ve learned at least half a dozen programming languages.

I never define myself as a programmer of any specific language, like Java programmer or C++ programmer - I just call myself a Software Development Engineer. Languages are merely tools; as long as I keep learning, I’ll naturally learn new programming languages when encountering new scenarios.

The computer world changes rapidly - new frameworks might emerge every few months, and new languages become popular every few years. Continuous learning is essential for maintaining competitiveness..

3 Independent Thinking

WeChat used to have a tradition of giving out annual gifts to employees.

In 2022, the annual gift we received was indeed an aluminum plate with “Keep Independent Thinking 2022” written on it.

The creator of WeChat has always emphasized the importance of “independent thinking” for WeChat, believing that if he had to choose the most important quality, he would choose “independent thinking.”

What superiors say isn’t necessarily right, what teachers say isn’t necessarily right, what academic institutions say isn’t necessarily right, what media says isn’t necessarily right, and loud voices certainly aren’t necessarily right - after all, being reasonable doesn’t require being loud.

For example, microservice architecture is very popular, and many companies are implementing microservices - does this mean monolithic architecture shouldn’t be used?

Should startups or small teams adopt microservice architecture for new businesses? Or should they start with monolithic architecture and migrate to services as the business grows?

Development inevitably involves various decisions, such as technology selection. For your requirements, you might find a dozen components that “seemingly” meet the requirements, and you might search online for evaluations of each component, finding diverse opinions. You need to independently analyze each component, identify their pros and cons, and make decisions based on your team’s characteristics.

Regarding independent thinking, my favorite quote is from the HBO miniseries “Chernobyl,” when scientist Valery Legasov asks the KGB to release his colleague Ulana Khomyuk, who was investigating the truth, saying he could guarantee she was no problem, the KGB chief responds:

Trust, but verify.

Because the first step of independent thinking is: start questioning.

4 Get It Running First

This phrase has a well-known variant: “It’s not like it doesn’t work.”

Many programmers are perfectionists, especially those who have read “Refactoring” and “Design Patterns,” who tend to spend a lot of time optimizing code and refactoring.

I used to have similar impulses, always wanting to spend time optimizing code, but after working on many projects, I have a strong feeling that it’s better to get the MVP launch first and let users experience it early.

If no users are using it, even the best and most beautiful code is meaningless.

So when I often see community members asking what language and framework to use for side projects, whether PHP/Python/Ruby would be too slow, my view has always been: build a prototype first, find the first user first.

When performance becomes a bottleneck, your business is already very success, and you’ll definitely have enough money to hire a dozen programmers to rewrite your project in Rust/C++, or assembly.

In this regard, I strongly agree with the my co-worker sitting next to me about code quality:

make it run, make it fast, make it beautiful.

Recently attempting side projects, I have a profound realization that technology might be the least important aspect of business.

Building a product from scratch and promoting it to users - users only care whether your product is useful and can solve their problems.

They don’t care whether you wrote it in C++/Java or JavaScript, nor do they care whether your code is elegant. Rather than obsessing over technology selection, it’s better to build the product first and let users try it.

5 The Most Comfortable Tool is the Best

I often see people in the community asking what’s the best language, best framework, best editor, best operating system.

“Best” is quite a subjective conclusion, and there’s no “best” solution for all scenarios, but I often see community members arguing over which language is better.

Or when someone shares about A, others reply that B/C/D is better, then arguments ensue.

This reminds me of the group identity phenomenon mentioned in “The Social Animal,” a famous social psychology work.

When fans develop strong identification with a team, they view the team as part of their self-identity, leading them to:

1. Use “we” instead of “they” to refer to the team
2. View the team’s success as personal success
3. React defensively to criticism of the team, viewing such criticism as attacks on themselves

If someone asks me this question, I answer “the tool you’re most comfortable and familiar with is best.”

Even if for fun, programming’s purpose is still to use computers to solve problems, and the best tool for solving problems is the one you’re most familiar with.

Unless the tool you know isn’t suitable for your problem, then naturally you need a new tool - don’t force a square peg into a round hole.

Of course, if it’s to satisfy curiosity and learn a new language, then choose what interests you.

When I learned Rust in 2017, it was simply because I had no classes in senior year, plenty of time, and wanted to learn something interesting and new. Rust 1.0 had only been released for 2 years then - I never expected to find work with Rust.

I can’t remember where I read this passage:

I once asked myself similar questions:

1. Do good things necessarily become popular? Not necessarily
2. Are things I like necessarily good things? Not necessarily
3. Will I spend time and energy on something that might not become popular but I like? Yes

6 Communicate More with People

While programmers certainly work with machines, fundamentally we’re still solving human problems.

When I first learned programming, I had a misconception that as long as I mastered the technology, I wouldn’t need to care about “interpersonal relationships.”

Therefore, after entering the workplace, I held such views and acted accordingly. While I wasn’t cold to others, I inevitably was, as friends described: “aloof.”

But after working for a long time, I realized that “interpersonal relationships” are inevitable - which is called networking and connections.

Even if my technical abilities are solid, I need to be seen by others. Having good relationships with colleagues and leaders allows for “win-win” when achievements are made.

So now I usually chat with colleagues whether there’s something specific or not, both to know each other better and learn more about the group, and to find potential optimization points from colleagues’ complaints, practicing my philosophy of “Work hard and be nice to people.”

After doing this job for a long time, I find out that software engineering is fundamentally human systems engineering.

7 Code Isn’t Everything

After writing programs for a while, it’s easy to have illusion that everything can be solved with code.

When you’re holding a hammer, everything looks like a nail.

The reality I learned is that many things cannot be solved with code. Code is just a tool that can only be used in appropriate scenarios - avoid path dependency.

Knowing how to write code isn’t useful, you also need to write articles, give presentations within the company, let others “see you.”

Professional ability in programming and project delivery is certainly important, but you also need soft skills in marketing yourself.

Chatting with the boss occasionally to increase communication and showing your visibility regularly might be more useful than launching ten projects.

8 Work with Excellent People

After years in the industry, having worked at Ant Group, WeChat, and AWS, and having worked with all kinds of colleagues, I have an increasingly strong insight:

Work with excellent people

Not only can you learn excellent qualities from them and improve technical abilities, you can learn best practices and engineering experience. During code reviews, you can learn better programming approaches, and when encountering problems, you have reliable teammates to help and guide you.

You’ll understand the uniqueness of systems developed by excellent programmers, know what simple and ergonomic systems look like, and develop your own technical taste.

Taste and aesthetics are abstract concepts, but after using good systems, you naturally won’t be interested in those crude, poorly made systems that rely on boss endorsement for forced promotion.

Improving technical taste, while enhancing our technical cognition, can in turn help us improve design capabilities.

Another benefit of working with excellent colleagues is building high-quality professional networks, which benefits career development and provides more options when changing jobs or switching tracks.

Although startups also have excellent developers, on average, Big Tech have higher proportions of excellent programmers, as they offer more competitive salaries and benefits, naturally having higher recruitment standards.

For example, WeChat has a so-called interview committee(like a group of bar raiser). Besides interviewers from the hiring department, candidates must also pass interviews by committee interviewers to avoid lowering standards for quick hiring.

So I personally suggest that new graduates should go to big tech when possible, to gain experience.

Although I left WeChat almost two years ago, I still miss the colleagues I worked with in the same group. They were truly technically excellent, super nice people, and willing to help.

9 Health Is the Foundation of Everything

After programming for so many years, I’ve developed a pile of occupational diseases.

I had tenosynovitis since university, developed a lower back pain after working for a few years and my once thick, dark hair is now increasingly sparse.

Because Tencent headquarters had a free gym, I basically went to the company gym on workdays to take advantage of company benefits - 2 days of cardio running, 2 days of anaerobic equipment training, persisting for almost 3 years.

I also started paying attention to my diet, trying to eat less oil and sugar and avoid alcohol.

While fitness isn’t a cure-all, at least I feel charged to handle high-intensity work.

Only when I lose something do I learn to cherish it. Only when I start taking medication and going to hospital for follow-ups do I start paying attention to my body.

Although programming is fun and supporting family is important, I still need to pay attention to my body. After all, health is the foundation of everything - if it breaks down, there are no other exciting stories.

10 Summary

Whether it’s programming or other skills, I feel they all follow the “Matthew Effect” - the more I learn, the more I understand, and the faster you’ll learn new things.

After writing a lot of code, I realized that a programmer’s competitiveness isn’t writing code, nor is it any particular language or framework. The core competitiveness is the ability to solve problems through technology - why be constrained by any specific programming language or technology?

I hope ten years of programming is just a starting point, and in ten years I can write another piece: “Reflections on Twenty Years of Programming.”

or using GitLens blame in VSCode:

Even though GUI tools is intuitive, but the Git CLI has much more powerful tooling for finding something closer to the real story behind your code.

There are many scenarios that CLI is valuable, the first is ignoring the whitespace changes.

For example, if you formatted your C++ codebase with clang-format or Javascript codebase with prettier, you haven’t actually changed the codebase, but you’re the owner of tons of lines of code.

The git blame -w option will ignore these type of whitespace changes.

The other great option is -C which will look for code movement between files in a commit.

For example, if you refactor a function from one file to another, the normal git blame will simply show you as the author in the new file, but the -C option will follow that movement and show the last person who actually change those lines of code.

-C is extremely helpful when I need to find out the original author of some lines of code after file renames or refactors, to know more about the background and context behind this code

According to the git blame doc, you could pass -C up to three times to ask Git try even harder:

1
2
3
4
5

-C[<num>]
           In addition to -M, detect lines moved or copied from other files that were modified in the same commit.
           This is useful when you reorganize your program and move code around across files.
           When this option is given twice, the command additionally looks for copies from other files in the commit that creates the file.
           When this option is given three times, the command additionally looks for copies from other files in any commit.

(it’s a bit of odd design)

Let’s take the access.rb file of ActiveModel module in Rails framework for example:

1

git blame activemodel/lib/active_model/access.rb

Ok, it looks like Jonathan Hefner wrote all of this code it appears, let’s look at the same code with git blame -w -C -C -C activemodel/lib/active_model/access.rb

Now we can see that Git has followed this code from file to file over the course of multiple renames, it turns out Jonathan Hefner is the most recent file renamer, Guillermo Iguaran is the original author.

If we want to know the history about this file, it’s much better to ask Guillermo rather than Jonathan, which is beyond what the GUI blame or normal Git blame tool reveals

1
2

git config --global user.name "Ramsay Leung"
git config --global user.email ramsayleung@gmail.com

The above command will simply add the user.name and user.email value into your ~/.gitconfig file

1
2
3
4
5
6
7
8

> cat ~/.gitconfig
[user]
    name = Ramsay Leung
    email = ramsayleung@gmail.com
[core]
    quotepath = false
[init]
    defaultBranch = master

You could also specify --local argument to writes the config values to .git/config in whatever project you’re currently in.

If you need to simultaneously contribute to your work and open source project on the same laptop, with different Git config values, e.g.(company email address for work-specific projects, personal email address for open source project), what should you do?

You could definitely set up work-specific config as global config, then set up personal config with --local for every personal project separately. It works, but tedious and easy to mess-up.

Fortunately, starting from Git version 2.13, Git supports conditional configuration includes, you are capable of setting up different configs for different repositories.

If you add the following config to your global config file:

1
2
3
4
5

[includeIf "gitdir:~/projects/oss/"]
    path = ~/.gitconfig-oss

[includeIf "gitdir:~/projects/work/"]
    path = ~/.gitconfig-work

Then Git will look in the ~/.gitconfig-oss files for values only if the project you are currently working on matches ~/projects/oss/.

Caution: If you forget to specify the “/” at the end of the git dir, e.g. “~/projects/oss”, Conditional Config won’t work!

Therefore, you could have a “work” directory and work-specific config here and an “oss” directory with values for your open source projects, etc.

Git also supports other filters more than gitdir, you could specify a branch name as an include filter with onbranch

1
2
3
4

  ; include only if we are in a worktree where foo-branch is
; currently checked out
[includeIf "onbranch:foo-branch"]
        path = foo.inc

Check out the Git docs for more details

As I farewelled to 2023, a year marked by numerous changes and personal evolution, I find myself recollecting the multitude of experiences that unfolded.

My 2023 journey was nothing short of fascinating and exciting, prompting me to revisit the year from various angles.

After seeing hoards of posts in social media generated by Github Contributions Chart, I thought I could also build an APP to summarize my Github contribution for every year for friends to have fun.

I spent my entire 4-days-new-year vocation to build this app named: Github Summary.

This project led me through a series of first-time experiences: first time to try Tailwind Css framework, first time to use and deploy project on Vercel, first time to build project on nextjs, first time to develop a public project on React(yes, I’ve tried to learn React for hundreds of times, but never get a chance to use it in real project), etc.

2 Happy 2024

While I hoped I could have completed this project by the close of 2023 to share summaries with friends, life’s timeline had other plans.

Now, as we step into 2024, I am thrilled to publish the GitHub Summary.

It’s never too late to showcase creative work, and this project is poised to generate insightful summaries not just for the past year but for the adventures that await in 2024.

Wishing everyone a Happy New Year! Feel free to explore GitHub Summary: https://github-summary.vercel.app/

1.1 IaC

Infrastructure as code(IaC) is the managing and provisioning of infrastructure through code instead of manual processes, for example, clicking button, adding or editing roles in AWS console.

1.2 AWS CloudFormation

AWS CloudFormation is the original IaC tool for AWS, released in 2011, which uses template files to automate and mange the setup of AWS resources.

1.3 AWS CDK

AWS Cloud Development Kit(CDK) is a product provided by AWS that makes it easier for developers to manage their infrastructure with familiar programming languages like TypeScript, Python, Java, etc.

And, CDK is standing on the shoulder of Cloudformation, providing tools for developers by leveraging Cloudformation.

A stack is a collection of AWS resources that you can manage as a single unit, like a box.

For instance, this box could include all the resources required to run an application or Lambda service, such as S3 Buckets (storage), Roles (authorization), Lambda Function (computing), API Gateway (access point), Alarm, Monitoring, etc.

2 Problem

I am currently working on a project which requires to set up two stacks, one stack( GlueStack ) for defining a list of AWS Glue tables and the other stack( ServiceStack ) for definition of Lambda service and associated resources.

In fact, S3 bucket names have to be globally unique within a partition, which means crossing the whole AWS customer base.

You are unable to create a S3 bucket with bucket name which is in use by another AWS customer or your own account.

So it’s safer to let CloudFormation generate a random bucket name for a developer when he need to initialize a S3 bucket.

However, there is new a problem I face: since the S3 bucket name is randomly generated characters, if GlueStack need to read the bucket created by ServiceStack, how could I share the bucket name between two stacks?

While these two stacks are isolated and separated, resources collection.

3 Solution

Fortunately, CDK offers a facility named CfnOutput to export a deployed resource, so that the consumer of the resource is able to Import required resource.

1. Define the required resource in ServiceStack (producer), for instance, a S3 bucket:

   1 2 3

   import { Bucket } from 'aws-cdk-lib/aws-s3'; const s3Bucket = new Bucket(this, 'MyBucketId', {});

2. Export the resource by specifying the value and exportName:

   1 2 3 4 5 6 7

   import { CfnOutput } from 'aws-cdk-lib'; // export the generated bucket name to other stack new CfnOutput(this, 'exportRequiredS3Bucket', { value: s3Bucket.bucketName, exportName: 'exportRequiredS3Bucket', });

3. Import the required resource in GlueStack (consumer):

   1 2 3

   import { Fn} from 'aws-cdk-lib'; const requiredS3BucketName = Fn.importValue('exportRequiredS3Bucket');

If we take a closer look at the synthesized CFN template for ServiceStack, we could find:

1
2
3
4
5
6
7
8
9

"Outputs": {
    "exportRequiredS3Bucket": {
        "Value": {
            "Ref": "MyBucketId737FC949"
        },
        "Export": {
            "Name": "exportRequiredS3Bucket"
        }
    }

The synthesized CFN template for GlueStack:

1
2
3

{
    "Fn::ImportValue": "exportRequiredS3Bucket"
}

This is the way about how to share value between two stacks.

4 Loose couping solution

Updated on 2023-12-02

People learn from mistake.

After applying this practice in my project, I recently learn that it’s not good practice to share resource across stack.

With using export/import, I tightly couple my stacks with a commitment that I can never update that unless I remove that couping later on.

It means it will become a disaster¹ whenever I need to update/delete the S3Bucket, CloudFormation will raise an error, complaining something like: “ServiceStack cannot be deleted as it’s in use by GlueStack”.

A better practice I learnt is adding a loose couping between ServiceStack and GlueStack by sharing a constant variable:

1. Define a constant variable somewhere:

   1 2 3

   export const Constants = { MyBucketName: 'TestBucket' }

2. Refine the definition of s3Bucket

   1 2 3 4 5

   import { Bucket } from 'aws-cdk-lib/aws-s3'; const s3Bucket = new Bucket(this, 'MyBucketId', { bucketName: Constants.MyBucketName, });

3. Refer the s3Bucket in GlueStack by MyBucketName instead of CDK exported reference

   1	const requiredS3BucketName = Constants.MyBucketName;

Therefore, these two stacks are not directly coupled, but they are referencing the same constant variable.

Then, CloudFormation won’t prevent you from updating the S3Bucket as there is not direct relation between these two stacks anymore.

This is the benefit of loose couping.

5 Reference

• API Document: class CfnOutput (construct)
• AWS Cloud Development Kit (AWS CDK) v2

In computer science, a topological sort or topological ordering of a directed graph is a linear ordering of its vertices such that for every directed edge uv from vertex u to vertx v, u comes before v in the ordering.

It sounds pretty academic, but I am sure you are using topological sort unconsciously every single day.

2 Application

Many real world situations can be modeled as a graph with directed edges where some events must occur before others. Then a topological sort gives an order in which to perform these events, for instance:

2.1 College class prerequisites

You must take course b first if you want to take course a. For example, in your alma mater, the student must complete PHYS:1511(College Physics) or PHYS:1611(Introductory Physics I) before taking College Physics II.

The courses can be represented by vertices, and there is an edge from College Physics to College Physics II since PHYS:1511 must be finished before College Physics II can be enrolled.

2.2 Job scheduling

scheduling a sequence of jobs or tasks based on their dependencies. The jobs are represented by vertices, and there is an edge from x to y if job x must be completed before job y can be started.

In the context of a CI/CD pipeline, the relationships between jobs can be represented by directed graph(specifically speaking, by directed acyclic graph). For example, in a CI pipeline, build job should be finished before start test job and lint job.

2.3 Program build dependencies

You want to figure out in which order you should compile all the program’s dependencies so that you will never try and compile a dependency for which you haven’t first built all of its dependencies.

A typical example is GNU Make: you specific your targets in a makefile, Make will parse makefile, and figure out which target should be built firstly. Supposing you have a makefile like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Makefile for analysis report

output/figure_1.png: data/input_file_1.csv scripts/generate_histogram.py
python scripts/generate_histogram.py -i data/input_file_1.csv -o output/figure_1.png

output/figure_2.png: data/input_file_2.csv scripts/generate_histogram.py
python scripts/generate_histogram.py -i data/input_file_2.csv -o output/figure_2.png

output/report.pdf: report/report.tex output/figure_1.png output/figure_2.png
cd report/ && pdflatex report.tex && mv report.pdf ../output/report.pdf

Make will generate a DAG internally to figure out which target should be executed firstly with typological sort:

3 Directed Acyclic Graph

Back to the definition, we say that a topological ordering of a directed graph is a linear ordering of its vertices, but not all directed graphs have a topological ordering.

A topological ordering is possible if and only if the graph has no directed cycles, that is, if it’s a directed acyclic graph(DAG).

Let us see some examples:

The definition requires that only the directed acyclic graph has a topological ordering, but why? What happens if we are trying to find a topological ordering of a directed graph? Let’s take the figure 3 for an example.

The directed graph problem has no solution, this is the reason why directed cycle is forbidden

4 Kahn’s Algorithm

There are several algorithms for topological sorting, Kahn’s algorithm is one of them, based on breadth first search.

The intuition behind Kahn’s algorithm is pretty straightforward:

To repeatedly remove nodes without any dependencies from the graph and add them to the topological ordering

As nodes without dependencies are removed from the graph, the original nodes depend on the removed node should be free now.

We keep removing nodes without dependencies from the graph until all nodes are processed, or a cycle is detected.

The dependencies of one node are represented as in-degree of this node.

Let’s take a quick example of how to find out a topological ordering of a given graph with Kahn’s algorithm.

Now we should understand how Kahn’s algorithm works. Let’s have a look at a C++ implementation of Kahn’s algorithm:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

#include <deque>
#include <vector>
// Kahn's algorithm
// `adj` is a directed acyclic graph represented as an adjacency list.
std::vector<int>
findTopologicalOrder(const std::vector<std::vector<int>> &adj) {
  int n = adj.size();

  std::vector<int> in_degree(n, 0);

  for (int i = 0; i < n; i++) {
    for (const auto &to_vertex : adj[i]) {
      in_degree[to_vertex]++;
    }
  }

  // queue contains nodes with no incoming edges
  std::deque<int> queue;
  for (int i = 0; i < n; i++) {
    if (in_degree[i] == 0) {
      queue.push_back(i);
    }
  }

  std::vector<int> order(n, 0);

  int index = 0;
  while (queue.size() > 0) {
    int cur = queue.front();
    queue.pop_front();
    order[index++] = cur;

    for (const auto &next : adj[cur]) {
      if (--in_degree[next] == 0) {
	queue.push_back(next);
      }
    }
  }

  // there is no cycle
  if (n == index) {
    return order;
  } else {
    // return an empty list if there is a cycle
    return std::vector<int>{};
  }
}

5 Bonus

When a pregnant woman takes calcium pills, she must make sure also that her diet is rich in vitamin D, since this vitamin makes the absorption of calcium possible.

After reading the demonstration of topological ordering, you (and I) too should take a certain vitamin, metaphorically speaking, to help you absorb. The vitamin D I pick for you (and myself) is two leetcode problems, which involve with the most typical use case of topological ordering – college class prerequisites:

• Course Schedule
• Course Schedule II

6 Reference

• Topological Sort | Kahn’s Algorithm | Graph Theory
• Directed Acyclic Graph
• Hands-on Tutorial on Make
• Topological sorting