Two weeks ago, while taking care of my child alone, I noticed a letter-tracing worksheet she brought home from daycare. It hit me that she was starting to grasp the concept of “homework.” In the United States, expecting a daycare to provide Chinese writing practice is almost a luxury. To help her build a foundational feel for writing Chinese characters, I decided to follow the pace of children growing up in China and create some practice sheets for her. After searching online, I found that most Chinese calligraphy worksheets are designed for older students or adults. For a child who can barely hold a pen steady, writing small characters is simply too challenging. I found a site called E-Zitie that offered exactly what I needed: large characters with stroke order guides. However, while the basic service is free, any customization requires a paid membership. Although the 38 RMB/year fee is negligible, the website’s UI felt like something from three generations ago, making me doubt its ability to meet my long-term needs.

Last week, a thought struck me: Why not build this tool myself? For a programmer with a background in infrastructure and backend development, building a web app is actually quite a daunting task. In my previous personal projects, the most time-consuming part was always the meticulous frontend tweaking. Even setting aside the frontend, architecting a backend that meets professional engineering standards is an exhausting endeavor. Whenever I thought about the sheer volume of work involved, my inner laziness would whisper: Just pay for the membership.

But this time, I started with an unusual level of confidence. My recent experiences using language models and practicing “Vibe Coding” (as coined by Andrej Karpathy) taught me that AI is already incredibly capable of building prototypes. So, during the gaps in my parenting duties, I wrote a 300-word requirement and design doc. What followed was a highly efficient dialogue: about 10 minutes of AI generation and another 10 minutes of fine-tuning. Gemini-CLI, powered by the latest Gemini Pro model, helped me stand up a clean, functional website in no time.

The project is currently hosted on my personal domain and can be accessed at toolbox.pengzhan.dev.

In less than 30 minutes, I was already printing out tracing sheets for my child to doodle on. This sense of instant gratification gave me a deeper realization of how AI will shape our future:

• Software creation is becoming a task anyone can perform, regardless of coding ability. This means that as long as you are willing to spend a few cents on LLM API calls, you can easily build what was traditionally considered a “program.” The barrier to entry for programming has dropped significantly. More importantly, when everyone can be a Product Manager, the question of “who implements it” becomes secondary. Take education as an example: if you are a physics teacher explaining force analysis, an editable PPT might capture your logic, but it’s hard to make it look right. Existing educational software might demonstrate the concept, but it might not align perfectly with your teaching style. So, why not let AI build a bespoke, web-based teaching demo just for you?

• Highly customized and truly innovative needs will define the human contribution to future software engineering. Since basic needs are now easily met, where does human value lie? Clearly, AI still struggles with unique, highly specific requirements or completely innovative explorations. Even when AI can “stitch together” a solution, the process is often laborious and requires constant human guidance. For instance, because AI still lacks precision in visual positioning and coordinate handling, the generated PDF had a persistent offset bug that AI couldn’t fix. Despite my lack of familiarity with the gopdf library, I had to dive into the source code myself to identify the error in 2D coordinate calculations. This “last mile” limitation is the current bottleneck of AI. Similarly, for completely unknown technical problems—like a Unix Socket error I encountered last week—LLMs (at least the latest Gemini-3.1-Pro) still cannot find a breakthrough using purely statistical logic. That human “spark”—an efficient activation of massive, compressed knowledge—remains far beyond the reach of current AI architectures.

• As LLMs enter the next phase and supporting tools mature, the very form of education will undergo a massive transformation. The power of LLMs is both transformative and “disruptive.” My initial goal was just to satisfy a need for writing practice, but the success of this tool made me realize the possibilities are much broader. Recently, I noticed my child struggling with counting worksheets from school; she often didn’t understand the instructions, leading her to color the wrong items or fail to fill in the counts correctly. So, I customized a new feature for her: I let the AI randomly generate similar counting exercises. This approach—completely on-demand and randomly generated—not only reinforces the skill but also prevents “false mastery” caused by repeating the same static problems. This is the true cornerstone of “student-centered” personalized education: it’s no longer just a theoretical ideal, but a tangible infrastructure we can finally build.

We cannot predict exactly what comes next. But I stand in awe of this wave of change, and I am deeply excited to witness the next evolution of human society.

Another pain point is photos. Every time I consider organizing them — deleting the meaningless ones and sorting the rest — I get overwhelmed by the sheer size of the task and put it off.

This is exactly the kind of tedious work AI is great at. Unlike more creative tasks where large language models (LLMs) can hallucinate, organizing files is straightforward and rules-based. It felt natural to me that AI should be able to help. For a long time, though, I didn’t know how to get started.

When agent modes became mainstream in AI platforms, I thought I had found the answer. But after trying ChatGPT’s web-based agent, I was shocked at how clumsy it was — watching it fail to click a simple checkbox in the Dropbox web client was painful. After wasting two hours, I gave up.

That’s when I started looking into the MCP protocol and realized it could help. Unlike GUI-based agents that burn tokens mimicking mouse clicks, MCP makes everything machine-readable and structured. After a bit of research, I decided to try Gemini-CLI (which recently topped GitHub’s trending charts) together with a third-party Dropbox MCP server.

Quick Setup #

Installing Gemini-CLI #

This part is straightforward. Just follow Gemini-CLI’s GitHub README. I tested on my MacBook and simply used Homebrew to install it:

brew install gemini-cli

After installation, you need to authenticate Gemini-CLI. You can either:

• Run gemini-cli, type /auth, then select 1. Login with Google

• Or use your Gemini API Key (see the official tutorial)

Running MCP Server #

Instructions for the Dropbox MCP server can be found in its README, or you can follow these steps:

git clone https://github.com/amgadabdelhafez/dbx-mcp-server.git
cd dbx-mcp-server
npm install
npm run build
npm run setup

During setup, you’ll need to create a Dropbox app in the Dropbox App Console. Make sure to grant all required permissions, then provide the details back to the interactive shell when prompted.

Once that’s done, the setup will open a browser window asking you to authenticate your Dropbox app. After login, you’ll be redirected to a local URL like this:

http://localhost/?code=WBjFx90NRQwdv8TM71kf343coS7Q5J2l8Wnqw

Copy the code back into your shell to complete the configuration. I recommend selecting 1. Generate Claude Desktop config when asked, which will automatically update your Claude configuration:

Would you like to generate MCP configuration files?
1. Generate Claude Desktop config
2. Generate Cline config
3. Generate both
4. Skip

Enter your choice (1-4): 1

You’ll then see confirmation that the Claude Desktop config has been updated.

Configuring Gemini for MCP Servers #

Now copy the MCP configuration into Gemini-CLI’s settings.json file located at ~/.gemini/settings.json. Make sure to allow the dbx-mcp-server in the mcp section, for example:

{
  ...
  "mcp": {
    "allowed": [
      "dbx-mcp-server"
    ],
    "excluded": []
  },
  "mcpServers": {   // Copied from Claude Desktop config
    "dbx-mcp-server": {
      "command": "node",
      "args": [
        "/tmp/Github/dbx-mcp-server/build/src/index.js"
      ],
      "env": {
        "DROPBOX_APP_KEY": "...",
        "DROPBOX_APP_SECRET": "...",
        "DROPBOX_REFRESH_TOKEN": "..."
      },
      "disabled": false,
      "autoApprove": []
    }
  }
}

Finally, restart Gemini-CLI. Run the command /mcp list and you should see the dbx-mcp-server listed as ready, with its available tools:

Configured MCP servers:

  🟢 dbx-mcp-server - Ready (13 tools, 2 prompts)
    Tools:
    - copy_item
    - create_folder
    - delete_item
    - download_file
    - get_account_info
    - get_file_content
    - get_file_metadata
    - get_sharing_link
    - list_files
    - move_item
    - safe_delete_item
    - search_file_db
    - upload_file

How I Use It #

I’m not going to dive deep into how MCP works — I’m a newbie myself. But I did find this blog post extremely easy to follow, especially with its self-explanatory flow chart:

Below are some of the prompts I’ve tried and what I learned from them.

Prompt 1 #

Prompt: Could you analyze all my major folders (top 3 levels) and give me a new directory tree structure I can further use?

Result:

• ❌ The folder suggestions still looked strange and needed a lot of guidance.

Prompt 2 #

Prompt: There are a lot of unorganized files in this folder. Could you categorize them in a general way that avoids overcomplicating or fragmenting into too many folders or nested folders? After that, generate a plan on how to move them and ask for my approval before execution.

Result:

• ✅ The assistant generated a plan and asked for feedback.

• I provided additional suggestions:

Took ~20 conversation rounds before it could reliably categorize based on file content.

• ⚠️ Privacy note: This prompt may leak personal information to the LLM service. Be cautious.

• Time: ~20 minutes of planning, ~400 files categorized. (Execution time not measured — I multitasked while it ran.)

Prompt 3 #

Prompt: Could you scan all non-target folders, find empty ones, list them, and then delete them after my review?

Result: ✅ Successfully executed and correctly removed empty folders.

Prompt 4 #

Prompt: Find all folders that could be merged based on name similarity (and path).

Result: ✅ Found 16 folder pairs that could be merged.

Personal Feelings and Conclusion #

Even though MCP + Gemini-CLI can achieve very good performance on some tasks, it’s still difficult for even the most advanced LLMs to fully understand situations and consistently make the right calls based on their own analysis. Human input is still essential.

Beyond that, I had five strong personal takeaways:

Prompt size limitations Even with a 2M token context, it’s tiny compared to human memory. Using flat, non-hierarchical memory structures for LLMs is not scalable — both logically and infrastructurally.

MCP’s verbosity While MCP is a solid connector schema, it’s verbose and sometimes makes the context size problem worse.

Good for general tasks, risky for details This approach works great for general asks, but struggles with detailed organization tasks like renaming or fine-grained document sorting. For example, it once miscategorized some of my identity documents (an uncommon format) into a tax folder — a serious mistake.

Weak multimedia handling Feeding in entire videos, large photo sets, or even a bulk of small photos from one trip doesn’t work well. This reminded me of how most YouTube summarizers today are still transcript-based rather than visual-based.

Privacy concerns Privacy is a major pain point. Even when using my own API key or self-deployed services, the risk of leaking sensitive personal information during file organization (or similar workflows) still feels dangerous.

Overall, MCP + Gemini-CLI is powerful and promising for Dropbox organization, but it’s not a complete replacement for careful human oversight. For now, I see it as an assistant — not an autopilot.

Debugging an unexpected container is feasible for self-hosted Kubernetes, where engineers have better observability and testing capability. However, if you are using GKE or another cloud provider’s Kubernetes engine, the situation can become complicated.

Here’s a problem I recently faced.

The Challenge: Debugging a Crashing vLLM Container on GKE #

Recently, I deployed a Large Language Model (LLM) model serving container on GKE as a Deployment. When I ran the serving framework in a non-containerized environment, everything worked perfectly, but it consistently crashed when running inside a Kubernetes pod.

The most challenging aspect of the situation was that no application stdout or logs were being generated by the application. The only information I had was from kubectl, which showed that the container had exited with status code 1.‘

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
meta-deployment-789d98c8f7-4fsz4 0/1 CrashLoopBackOff 5 10m

To better understand why this issue was happening, I tried the same container with different entry points and commands. I found that the issue was highly related to the keyword “vllm”. Every time a command contained the string “vllm”, it would crash (except when I used echo vllm). It seemed that echo worked because it’s a shell built-in command, but any command that involved exec crashed.

With this context, I decided to use gdb it to debug the container. To do that, I needed to rebuild the container image on top of the existing one. However, even running gdb --args python -m vllm.entrypoints.api_server .... still resulted in a crash. To move forward, I decided to rely on strace, a kernel tracing utility, to understand what was happening.

Here’s the tricky part: GKE’s default worker node OS image is COS (Container-Optimized OS) , a Debian-based Linux OS with many restrictions. For example, it lacks binary/package managers, and most paths under its file system are non-executable. So, how could I easily download and use strace?

Step-by-Step Guide: Debugging GKE CrashLoopBackOff with Strace #

Since we were already on a containerized platform, the simplest approach was to deploy another container and perform all operations within it. This container also needed to fulfill three criteria:

• The strace pod must be privileged.

• The strace pod must have the SYS_PTRACE Linux capability.

• The strace pod must be in the host PID namespace to view all processes.

Here’s an example of pod I used

$ cat &1 | tee strace.log # pid to the target container, e.g. 4128

# Go back to target pod's shell
root@meta-deployment-789d98c8f7-4fsz4:/$ python -m vllm.entrypoints.api_server ....

# Inside strace pod
wait(-1, 
fork(....

After that, I was able to examine the strace log to understand why and when my container was crashing.

Summary #

In this post, I explained a complicated scenario and the limitations of GKE when debugging it. I also described the solution to overcome these debugging difficulties by enabling the powerful strace utility on a GKE node. As for the reason behind this strange behavior, that will be discussed in a later post. It’s a different but interesting problem.

Cross-references #

[1] vLLM deployment example from GCP Vertex Model Garden

apiVersion: apps/v1
kind: Deployment
metadata:
  name: meta-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: meta-server
  template:
    metadata:
      labels:
        app: meta-server
        ai.gke.io/model: Llama-3-1-8B-Instruct
        ai.gke.io/inference-server: vllm
        examples.ai.gke.io/source: model-garden
    spec:
      containers:
      - name: inference-server
        image: us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-vllm-serve:20240821_1034_RC00
        resources:
          requests:
            cpu: 8
            memory: 29Gi
            ephemeral-storage: 80Gi
            nvidia.com/gpu : 1
          limits:
            cpu: 8
            memory: 29Gi
            ephemeral-storage: 80Gi
            nvidia.com/gpu : 1
        command:
        args:
        - python
        - -m
        - vllm.entrypoints.api_server
        - --host=0.0.0.0
        - --port=7080
        - --swap-space=16
        - --gpu-memory-utilization=0.9
        - --max-model-len=32768
        - --trust-remote-code
        - --disable-log-stats
        - --model=gs://vertex-model-garden-public-us/llama3.1/Meta-Llama-3.1-8B-Instruct
        - --tensor-parallel-size=1
        - --max-num-seqs=12
        - --enforce-eager
        - --disable-custom-all-reduce
        - --enable-chunked-prefill
        env:
        - name: MODEL_ID
          value: "meta-llama/Llama-3.1-8B-Instruct"
        - name: DEPLOY_SOURCE
          value: "UI_NATIVE_MODEL"
        volumeMounts:
        - mountPath: /dev/shm
          name: dshm
      volumes:
      - name: dshm
        emptyDir:
          medium: Memory
      nodeSelector:
        cloud.google.com/gke-accelerator: nvidia-l4
---
apiVersion: v1
kind: Service
metadata:
  name: meta-service
spec:
  selector:
    app: meta-server
  type: ClusterIP
  ports:
  - protocol: TCP
    port: 8000
    targetPort: 7080

As I hit this two-year milestone, I wanted to share my reflections on working in this field. This isn’t specific to GKE but more about Kubernetes as a whole. I’m still a relative newbie, but here are some stereotypes, myths, and pain points I’ve encountered:

The Customization Conundrum: Complexity is King #

Kubernetes was designed to orchestrate containerized workloads, scaling them and allocating resources seamlessly. But over its ten years, Kubernetes has evolved into a giant, complex beast. Years of user requests and feature additions have made it adaptable to various scenarios, but at the cost of simplicity.

One example that suddenly comes to mind is container types. Apparently, there are two types of containers in nature. InitContainer is for the setup work of a pod before the main container runs. The main container is just the workloads that are containerized to be run to serve users’ needs. And after that, Ephemeral containers and sidecar containers (a special type of initContainer) were introduced for some special needs. These changes significantly increase K8s container lifecycle’s customization, but at what cost? I guess that the pod lifecycle becomes more complex, and new features will only serve some complex cases, which will confuse “orthodoxy” users.

The Dream Gap: Kubernetes for the Big Leagues #

Kubernetes are so hard for small groups and single users. Even miniKube is way too complex! I believe everyone who touches Kubernetes, no matter newcomers or experts, is complaining about using YAML to write all workloads. I’ve seen some YAML with thousands of lines of code that still does not include custom resources. Honestly, even writing the spec YAML files will push back all tasters.

Since the K8s were originally designed for orchestration problems targeting large systems, we shouldn’t complain that only big companies are using them. However, the story changed when time flew by. For all K8s users, the existing system already satisfied most of their basic needs, but situations are totally different for bigger companies, they always have some new requirements and lead the development of the K8s OSS. This turns ugly when different cloud providers start a differential competition. Due to lower-level compute resource differences, more and more optimizations for K8s are becoming secrets. Companies started to keep their own code base and only merge necessary codes upstream. This makes the system hard to understand, not even mentioned as clean and elegant.

The Infrastructure Investment Illusion #

Most Kubernetes users who rely on cloud providers have the same strange thinking that they can cut off investment in infrastructure. I feel the logic behind this is that users always believe that all orchestration will be done by a cloud provider, they should save time and resources by maintaining their own infrastructure. This turns into completely wrong thinking. All top customers use Kubernetes Engineers provided by the cloud provider but also keep their own tech stack and experts. Sometimes I also don’t understand why they don’t build on top of the cloud provider’s bare-metal machines, since they have all the experts and knowledge on K8s.

But things are completely different for some large-cap but non-tech companies. Instead of infra experts, they relied on hiring “expert K8s users.” They can definitely help with deploying workloads and doing some simple debugging. However, once a major issue impacts them, they can do nothing but wait for cloud providers to support them. To their mindset, using Kubernetes is investment-transferring, we paid slightly more rather than buy down the cloud VMs, but instead, they asked the cloud provider to support them. Sadly, even cloud providers can’t provide better support, and I observed the trend that those large-cap non-tech companies started to hire K8s developers to counter this. Overall, it still becomes costly.

These are my reflections after two years. With deeper understanding, I’m sure my perspective will evolve. As I spend more time in this domain, I hope to share insights on K8s’ evolution and projects that might bridge the gap towards a more accessible Kubernetes.

Also, please share your thought with me.

Minimalism is the art of knowing how much is just enough. Digital minimalism applies this idea to our personal technology. It’s the key to living a focused life in an increasingly noisy world.

Cal Newport. Digital Minimalism

What’s the most valuable idea if we have truly advanced AI technology? I bet personalized assistants will be one of the top usable use cases. Back in years, when watching White Christmas, one episode from the TV series Black Mirror, I felt so astonishingly that screenwriters were imagining the same as I do. A purely personalized assistant (no spoiler) would think as the user does, make some decisions as the user thinks, and improve as the user experience grows.

Unfortunately, this is still early! But LLM opens a small gap that we can peek at.

In my previous post, I mentioned trying to visualize the keywords from the top news stories of the week. It was a good way to quickly scan a bunch of content, but it wasn’t ideal for understanding the details of any single piece.

Lately, I’ve been focusing on using Large Language Models (LLMs) to summarize long articles into something more readable, especially for non-native speakers. And guess what? Today I found a cool way to do it using Apple’s App Shortcuts on iOS!

Here’s a demo:

Ready to give it a shot? Just install the ChatGPT and Claude iOS apps before using the shortcut with this link: https://www.icloud.com/shortcuts/291a21bb62ea431a937dcbd9b21218fc.

The whole process #

Here’s the shortcut’s entire process to achieve what you saw in the above demo (be warned, it’s a long screenshot!). To give you a better understanding before we dive in, let’s break down the whole process with some bullet points.

• 1. Grab the URL: The shortcut retrieves the URL from your device’s default share menu.

• 2. Choose Summarization Language (Optional): If you’re a polyglot (someone who speaks multiple languages), you can use a pop-up menu (also called a switch statement in programming) to choose between summarizing in English or Chinese. Skip this step if you only need English summaries.

• 3. Fetch Web Content: The shortcut uses the Safari engine to grab the content of the web page associated with the URL.

• 4. Prepare the Summary Request: The shortcut combines the chosen summarization language (if applicable) with the fetched web content and copies the combined text to your clipboard.

• 5. Select LLM Provider App: Finally, a pop-up menu allows you to choose which Large Language Model (LLM) provider app you want to use for the summarization.

Some tips for using scripting blocks #

Sadly, Apple seems to have given up on this feature. Documents are lacking, engagement is low making it hard to find answers. Here is the problem I encountered.

“The web content could not be loaded because your privacy settings do not allow this shortcut to load external web content. You can change this in the Privacy tab of the shortcut’s settings.“

The error message suggests fixing privacy settings, but clicking “More” (middle button in the bottom tab) reveals it’s not related. Here’s my guess: “Set Variable” likely creates a reference (alias) to the content, not actually changing its data type. The “Combine” block might not work with this “web of content” type.

The solution is simple (as shown in the screenshot): Use a “Text” block instead. Include the variable you want to combine, and it will convert it to plain text before combining. This creates a new text variable for the “Combine” block to work with.

Symptom #

Recently, I faced an issue where Kubelet on a node reported error message failed to create file descriptors.

error creating file watcher: too many open files
error creating file watcher: no space left on device

After short checking, I found the node has max_user_watches of 10000, but the TotalinotifyWatches is beyond this value. (P. S still not sure why watcher can initiate more than cap). In order to find which process occupied the most watchers. I used following command^[1] to find it out.

echo -e "COUNT\tPID\tUSER\tCOMMAND"; sudo find /proc/[0-9]*/fdinfo -type f 2>/dev/null | sudo xargs grep ^inotify 2>/dev/null | cut -d/ -f 3 | uniq -c | sort -nr | { while read -rs COUNT PID; do echo -en "$COUNT\t$PID\t"; ps -p $PID -o user=,command=; done}

COUNT   PID USER    COMMAND
7491    8412    root /home/kubernetes/bin/kubelet --v=2 --cloud-provide=gce --experi
2620    1   root /sbin/init
....

Surprisingly, Kubelet initiated more than 7000 inotify watchers. I assumed there was an inotify leakage in Kubelet.

Leakage check #

Clean Kubelet #

To better understand the situation, I created a clean cluster with only 1 clean node on GKE. Roughly 70 inotify watchers were there. I created a single nginx pod and the number increased by 3. Theoretically, these 3 watchers are used by Kubelet to monitor any changes on rootfs, kube-api-access and PodSandbox. But to verify it, we need to check more details on which inodes are monitored by Kubelet.

Check inotify file descriptors #

To do so, let’s take a look how to track a single inotify file descriptor. Opened processes’ fdinfo folder, we can examine each or them to find an inotify fd.

# Find kubelet pid
ps -aux | grep kubelet
KPID=2430

# File the an example fd
sudo ls /proc/2430/fdinfo

0 1 10 11 12 13 14 2 3 4 5 6 7 8 9

...

sudo cat /proc/2430/fdinfo/8

pos:    0
flags:  02004000
mnt_id: 15
ino:    1057
inotify wd:1 ino:3f327 sdev:800001 mask:fc6 ignored_mask:0 fhandle-bytes:8 fhandle-type:1 f_handle:27f30300e5059ea2

This is very confusing, so I rely on man proc^man_proc to understand every piece of them. In given fd, the needed information to continue sit in the last line. It’s an inotify entry represents the 1 file or folder to be monitored. And the most useful data is ino:3f327 which represents the inode number of target file (in hexadecimal). And sdev:800001, which represents the ID of device where the inode sit on, and it’s also in hex.

Using lsblk, I can see there’s only 1 disk I’m using on the node, so finding the target file would be easy.

# Cast to decimal
ino=3f327
dec="$((16#${ino}))"

# Find the target file
loc="debugfs -R 'ncheck ${dec}' /dev/sda1"
sudo eval $loc 2>/dev/null

debugfd 1.46.5 (30-Dec-2021)
Inode   Pathname
258855  /etc/srv/kubernetes/pki/ca-certificates.crt

Put all processes above into one single script(please see the updated version in the appendix), I can retrieve all target files, that would help to understand if there’s a real leakage. Also, I count the unique inode amount, this could also help to know which inode are monitored multiple times.

cat /dev/null | xargs grep ^inotify | cut -d " " -f 3 | cut -d ":" -f 2)
echo ${in_fds}
echo "Count: $(find /proc/${PID}/fdinfo -type f 2>/dev/null | xargs grep ^inotify | wc -l)"

uniq_fds=$(echo "${in_fds[@]}" | sort | uniq)
echo ${uniq_fds}

while read -r element;
do
  count=$(echo "${in_fds[@]}" | grep -o "${element}" | wc -l)
  dec="$((16#${element}))"
  loc="debugfs -R 'ncheck ${dec}' /dev/sda1"
  loc=$(eval $loc 2>/dev/null | tail -1 | cut -d " " -f 4)
  printf "%-6s %-10s %-6s %s\n" "${element}" "${dec}" "${count}" "${loc}"
done /dev/null | xargs grep ^inotify)
# echo ${inums_raw}
echo "Count: $(find /proc/${PID}/fdinfo -type f 2>/dev/null | xargs grep ^inotify | wc -l)"

while read -r line;
do
        reg="ino:([0-9a-f]*) sdev:([0-9a-f]*)"
        if [[ ${line} =~ $reg ]]; then
                ino="${BASH_REMATCH[1]}"
                sdev="${BASH_REMATCH[2]}"
                # echo $ino $sdev
        else
                echo "wrong line"
        fi

        sdev_in_dec=$((16#$sdev))
        minor=$((sdev_in_dec % 256))
        major=$((sdev_in_dec / 256))
        # echo "${major}:${minor}"

        in_fds_sdev+=("${ino}-${major}:${minor}")
done <<< "${inums_raw}"

uniq_pairs=($(echo "${in_fds_sdev[@]}" | sort | uniq))
echo "Unique target" ${#uniq_pairs[@]}

printf "%-10s %-10s %-6s %s\n" "INUM" "DEV" "COUNT" "TARGET"
for pair in "${uniq_pairs[@]}"
do
        count=$(echo "${in_fds_sdev[@]}" | grep -o "${pair}" | wc -l)
        fd_hex=$(echo ${pair} | cut -d "-" -f 1)
        dev=$(echo ${pair} | cut -d "-" -f 2)
        fd_dec="$((16#${fd_hex}))"

        mount_info=$(grep ${dev} /proc/$PID/mountinfo)
        if [[ -z $mount_info ]]; then
                echo "Can't find mount info for" $dev
        else
                tmpfs_path=$(echo $mount_info | cut -d " " -f 5)
                # echo $tmpfs_path
                loc=$(find ${tmpfs_path} -inum ${fd_dec})
                printf "%-10s %-10s %-6s %s\n" "${fd_dec}" "${dev}" "${count}" "${loc}"
        fi
done

References #

Due to its central role, debugging Kubelet becomes crucial when troubleshooting your Kubernetes environment. In the post, we will continually expand methods on how to debug Kubelet, especially some practices on some real-world use cases.

Debug logs #

Like all others program’s debugging, the most straightforward way for newbies and the easiest way for advanced developer is relying on logs. Similar to debugging Kubelet, bumping up verbosity to show more logs is the most intuitive approach when facing an issue. Like most components in Kubernetes, kubelet uses klog for logging and there are 10 verbosity levels(0-9).

TL;DR: Bumping up to level 5 would satisfy most debugging needs.

LevelMeaningExample0Always on (Warning, Error, Fatal)Example1Default level logs when don’t want any verbosityExample2Most important logs when major operations happen, also the default verbosity levelExample3Extended informationExample4Debug levelExample5Trace levelExample6Display requested resourcesExample7Display HTTP request headersExample8Display HTTP request payloadExample

By the time, this note was written. In kubelet related code, level 8 was only used pkg/kubelet/prober/prober_manager.go and level 7 was only used in pkg/kubelet/logs/container_log_manager.go. 11 occurrences in level 6 were used, and all of them are not part of workload lifecycle related.

Add trace cide #

Kubelet with Bpfcc-tools #

Debugger #

Further readings #

Inotify watcher leaks in Kubelet

The reason why I am writing this down is that it has been a week and no students have asked for the solution for the last Lab. I realize that the learning gap between students is huge, especially when a non-profit university is admitting more and more students. To help all students understand the concepts of modern OS, I decided to write this post.

It starts with the past lab content I have (as the skeleton), and will be amended with extra materials I think it helps. Remember, it’s for help in learning. DON’T COPY & PASTE CODE!

Lab1-Introduction #

Lab3-Process #

Lab4-IPC #

Lab6-7-Scheduling #

First user process in xv6 #

Kernel works #

In xv6, as the same as conventional Linux OS, the very first user-level process is init. Before init’s running, all the OS bootstraps happen in a highly privileged mode(kernel level).

Xv6’s kernel has the entry point as the main function located in the file main.c. The main function invokes 17 functions to set up kernel page tables, interrupt handlers, I/O devices and etc. When all kernel preparations are done, by calling the function userinit(), the kernel will boot up process init.

int
main(void)
{
  kinit1(end, P2V(4*1024*1024)); // phys page allocator
  kvmalloc();      // kernel page table
  mpinit();        // collect info about this machine
  lapicinit();
  seginit();       // set up segments
  cprintf("\ncpu%d: starting xv6\n\n", cpu->id);
  picinit();       // interrupt controller
  ioapicinit();    // another interrupt controller
  consoleinit();   // I/O devices & their interrupts
  uartinit();      // serial port
  pinit();         // process table
  tvinit();        // trap vectors
  binit();         // buffer cache
  fileinit();      // file table
  ideinit();       // disk
  if(!ismp)
    timerinit();   // uniprocessor timer
  startothers();   // start other processors
  kinit2(P2V(4*1024*1024), P2V(PHYSTOP)); // must come after startothers()
  userinit();      // first user process
  // Finish setting up this processor in mpmain.
  mpmain();
}

It’s tricky since that init is a user process, but kernel can’t call any user-level system calls to create it. Why? 1. Kernel has all privileges to create a user process. So it doesn’t need to call system calls such as fork(). And 2. All other user processes can be created by forking from its parent. Forking including clone the whole user virtual memory layout. However, the first process has no parent to fork from. That’s why it makes the creation of the first user process becomes so unique.

In proc.c, userinit() define there gives us the whole procedure of creating init. Similar to the fork(), but more simple. Process control block(structures for storing the process status) was created at the very first by calling allocproc(). After then, by invoking setupkvm()(defined in vm.c), kernel memory map was setup for the process. During setting up kernel memory map, a page size virtual memory will be assigned to the process as ready. And later, this page size memory will be used to store instructions of init.

Followed by setup kernel stack for the init process, calling inituvm() will load init’s text into the page that is just being allocated. inituvm() takes 3 arguments: a pointer to the process’s page directory (p->pgdir), a char-type pointer declared from external which point to init’s text segment(_binary_initcode_start), and a char-type pointer which points to an external integer as the size of the init’s text segment(_binary_initcode_size). Simply put, it will load instructions of init into the memory.

So now, the problem becomes when and where did instructions for init have compiled into the kernel?

void
userinit(void)
{
  struct proc *p;
  extern char _binary_initcode_start[], _binary_initcode_size[];

  p = allocproc();
  initproc = p;
  if((p->pgdir = setupkvm()) == 0)
    panic("userinit: out of memory?");
  inituvm(p->pgdir, _binary_initcode_start, (int)_binary_initcode_size);
  p->sz = PGSIZE;
  memset(p->tf, 0, sizeof(*p->tf));
  p->tf->cs = (SEG_UCODE tf->ds = (SEG_UDATA tf->es = p->tf->ds;
  p->tf->ss = p->tf->ds;
  p->tf->eflags = FL_IF;
  p->tf->esp = PGSIZE;
  p->tf->eip = 0;  // beginning of initcode.S

  safestrcpy(p->name, "initcode", sizeof(p->name));
  p->cwd = namei("/");

  p->state = RUNNABLE;
}

Where the user-level code was integrated? #

If you search the keyword “_binary_initcode_start” in the source code, you can’t find any references. The clue comes from the Makefile.

In the makefile, initcode is a prerequisites to compile the kernel image. Step 1: Before kernel was compiled, initcode.S was first compiled to a runnable binary initcode. This binary was very odd because it was not supposed to let any other OS to run it. Initcode.s was first compiled without any standard including, and generating the intermediate file initcode.o.

Step 2: Initcode.o then linked to Initcode.out with two uncommon settings. First it specify the entry of this binary file as when “start” symbol points to. This “start” symbol was declared in the assembly code. Second it specify a absolute address(0) for the text segments. By doing this, text segments will be placed at the start of the binary file (except the header of the ELF)^[1].

Step 3: Initcode.out is already a minimized binary but it’s not enough. That’s why when using objcopy to copy it to the file initcode, it further strip all headers and debug information^[2]. At this point, we have a minimal binary file initcode. From the first byte of this file, it’s only includes runnable instructions. And the size of the file is only 44 bytes.

initcode: initcode.S
    $(CC) $(CFLAGS) -nostdinc -I. -c initcode.S                         # Step 1
    $(LD) $(LDFLAGS) -N -e start -Ttext 0 -o initcode.out initcode.o    # Step 2
    $(OBJCOPY) -S -O binary initcode.out initcode                       # Step 3
    $(OBJDUMP) -S initcode.o > initcode.asm

This binary later were appended to the kernel using following commands. And during this appending, 3 symbols were generated and added to the symbol table of the kernel^[1:1]. “_binary_initcode_start” contains the address of where the initcode segment was appended to. “_binary_initcode_end” contains the address of where the initcode segment was ended at. “_binary_initcode_size” is a ABS type symbol with value 0x2C(45) that specify the size of the initcode segment is 45 bytes.

kernel: $(OBJS) entry.o entryother initcode kernel.ld
    $(LD) $(LDFLAGS) -T kernel.ld -o kernel entry.o $(OBJS) -b binary initcode entryother #  kernel.asm
    $(OBJDUMP) -t kernel | sed '1,/SYMBOL TABLE/d; s/ .* / /; /^$/d' > kernel.sym

In short summary, using objdump, we can verify that source code initcode.S has been compiled and loaded into the kernel. Also the segment of initcode’s instructions was located by the pointer “_binary_initcode_start”. That’s explain when calling inituvm(p->pgdir, _binary_initcode_start, (int)_binary_initcode_size);, functionalities implemented in initcode.S will be loaded into the runtime of the first process within xv6.

# Header of the file kernel
kernel:     file format elf32-i386
kernel
architecture: i386, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x0010000c

Program Header:
    LOAD off    0x00001000 vaddr 0x80100000 paddr 0x00100000 align 2**12
         filesz 0x00008c6a memsz 0x00008c6a flags r-x
...
Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00008586  80100000  00100000  00001000  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
...
SYMBOL TABLE:
...
8010b50c g       .data  00000000 _binary_initcode_end
...
8010b4e0 g       .data  00000000 _binary_initcode_start
...
0000002c g       *ABS*  00000000 _binary_initcode_size
...

User-level code #

Take a look of content in the initcode.S, you will find the code can explain itself well. There are no other jobs but just calling system call exec to run a user-level binary “init”.

Initcode.S:

# Initial process execs /init.

#include "syscall.h"
#include "traps.h"

# exec(init, argv)
.globl start
start:
  pushl $argv
  pushl $init
  pushl $0  // where caller pc would be
  movl $SYS_exec, %eax
  int $T_SYSCALL

# for(;;) exit();
exit:
  movl $SYS_exit, %eax
  int $T_SYSCALL
  jmp exit

# char init[] = "/init\0";
init:
  .string "/init\0"

# char *argv[] = { init, 0 };
.p2align 2
argv:
  .long init
  .long 0

The “init” mentioned above is not a pure user-level binary executable that compiled from the source code init.c. Within init.c, a file named console will be created at the runtime for saving standard outputs and errors. Then it will forked a child process(the second user process), and let it run program “sh”.

“sh” is the xv6’s default shell, a user-level program that generated from source sh.c. After the shell boots up, you can interactive with the xv6. This’s how first process (and second process) was started in the xv6.

init.c:

// init: The initial user-level program

#include "types.h"
#include "stat.h"
#include "user.h"
#include "fcntl.h"

char *argv[] = { "sh", 0 };

int
main(void)
{
  int pid, wpid;

  if(open("console", O_RDWR) = 0 && wpid != pid)
      printf(1, "zombie!\n");
  }
}

Xv6’s round robin schduler #

The Scheduler is the core of an operating system. With the scheduling of processes, the kernel can achieve near-real-time execution of multiple workloads. The scheduling problem is also an active aspect of computer science research. You can’t have one algorithm to fit all scenarios.

Xv6 by default has a round-robin scheduler. It’s controlled using two-level for-loops, where the top-level for-loop is an endless loop that will keep the scheduler busy running. The second-level nested for-loop will iterate a data structure named Ptable where all control information for processes is stored. Information including pid, process name, etc. is stored in a structure called proc. Ptable is an array of processes. Every runnable process in the Ptable will run strictly 1 time tick until the for-loop reached the last process in the Ptable. Then it will loop back to the top-level for-loop for the next iteration of processes.

// In file proc.c
struct {
  struct spinlock lock;
  struct proc proc[NPROC];
} ptable;

// In file proc.h
struct proc {
  uint sz;                     // Size of process memory (bytes)
  pde_t* pgdir;                // Page table
  char *kstack;                // Bottom of kernel stack for this process
  enum procstate state;        // Process state
  int pid;                     // Process ID
  struct proc *parent;         // Parent process
  struct trapframe *tf;        // Trap frame for current syscall
  struct context *context;     // swtch() here to run process
  void *chan;                  // If non-zero, sleeping on chan
  int killed;                  // If non-zero, have been killed
  struct file *ofile[NOFILE];  // Open files
  struct inode *cwd;           // Current directory
  char name[16];               // Process name (debugging)
};

// In file proc.c
void
scheduler(void)
{
  struct proc *p;

  for(;;){
    // Enable interrupts on this processor.
    sti();

    // Loop over process table looking for process to run.
    acquire(&ptable.lock);
    for(p = ptable.proc; p state != RUNNABLE)
        continue;

      // Switch to chosen process.  It is the process's job
      // to release ptable.lock and then reacquire it
      // before jumping back to us.
      proc = p;
      switchuvm(p);
      p->state = RUNNING;
      swtch(&cpu->scheduler, proc->context);
      switchkvm();

      // Process is done running for now.
      // It should have changed its p->state before coming back.
      proc = 0;
    }
    release(&ptable.lock);

  }
}

It’s not hard to understand why this logic makes a round-robin manner. This is very important to understand how to pick a process to run because scheduling is about always picking the appropriate process to achieve higher performance.

You can always come up with some new ideas for designing a good scheduler policy. Understanding how to switch from one process to another is equivalently important.

Once the process for the next time tick is selected. It’s time to switch from the running scheduler to the selected process. Wait for a second, there are two questions we haven’t answered.

• What is the running scheduler?

• How did the last running process stop running and give the CPU back to the scheduler?

Lab #

How does data support us training on edge? #

Before designing and implementing our framework, we first need confirmation that training on edge resource-limited devices is worthwhile. We were using a malware detection neural network to show why a small, customized neural network is better.

We collected 32000+ mobile apps feature as global data. With these data records, we trained a multilayer perceptron called “PerNet” to determine whether a given feature belongs to a benign or malware app. We called this detection. As well, PerNet can also classify malware apps into different types of attacks. We called this classification. The global model can achieve 93% above recall rate and 96.93% above accuracy.

With all these data, we selected two community app usage sub-dataset for local model generations.

Large categories (Scenario 1) We chose the 5 largest categories of apps, including entertainment, tools, brain&Puzzle, Lifestyle, and Education, as well as the 5 largest malware categories. All together, 12000+ apps were included in this sub-dataset, almost 50 to 50 between benign and malware.

Campus-community categories (Scenario 2) We chose the 5 most downloaded categories from college students as benign groups, as well as a similar amount of 5 malware categories. To ensure that malware apps are included in 5 benign categories, we also considered synthesizing some other malware apps within categories of 5 most downloaded(benign) categories.

With these two types of sub-dataset, we used the same PerNet to generate multiple local models. Under each scenarios experiment, we compared global and local models on the preserved test dataset. In all classification performances, local beat global in every scenario. In detection performances, local also share the same accuracy as global does.

In summary, local models were trained on special occasions. Under the same circumstance, a global model can achieve no better accuracy than local models. The reason why local is better might be because of overfitting. I believe this issue also be considered in the machine learning communities that they brought transfer learning, a technique to optimize global models to special scenarios but performing more training to a global model once it’s shipped to local.

Design and Implementation #

Overall design #

The basic EDDL distributed training setup consists of 3 parts. EDDL training cluster, a device cluster that consists of edge or mobile devices that are participating in training. EDDL manager, the initial driver program that works as collect training data, relay data to training devices and initial training clusters. Training data entry (TDE), a data storage for all training data.

Dynamic training data distribution #

Existing distributed DNN training solutions usually statically partition training data among workers. It can be a problem when the training node joins and exits. We designed our framework that can dynamically distribute training data during learning. Before every training batch started, a batch of TDE will be sent to devices.

In our experiments, we found that by applying this design, overall training time was shortened by doing. Especially in large amount devices cases, this optimization can be 50% less than statically divided.

Scaling up cluster size #

Our framework was designed to have both sync and async parameter aggregation. Asynchronous aggregation can allow a high outcome of training batch but with a sacrifice or converge time. Synchronous aggregation allows a quick converge time in epochs, however can’t ensure performance when there’s a struggler worker.

As showed in experiments, we chose sync as default because the converging time is dominant in overall training time. But, we also considered the possibilities of that async with more workers can achieve similar overall training time.

We introduced a formula to determine whether adding more training nodes can help or not. Here we used bandwidth usage coefficient (BUC) as

$$ BUC = \dfrac{n}{T_{sync}} $$

In this formula, $$n$$ is the number of devices, and $latex T_{sync}$ is the transmission time of parameters. With an increasing number of workers, n increase linearly but transmission time does not. When $$BUC$$ increases, the cluster can speed up training time by adding workers. Otherwise, adding more workers won’t help with overall training time.

Adaptive leader role splitting #

The idea of role splitting is simple that a device can work as a worker as well leader. The advantage of doing this is straightforward that we can transfer 1 less parameter and training time will be shortened.

However, in our current settings, it can’t perform much better help since only 1 leader role is in a cluster. We can benefit from this in our future works.

Overall architecture #

Details were given in the image.

Prototype hardware and software #

EDDL was designed to be run on two single-board computer embedded platforms. One such platform is ODROID-XU4, which is equipped with a 2.1/1.4 GHz 32-bit ARM processor and 2GB memory. The other platform is the Raspberry Pi 3 Model B board, which comes with an ARM 1.2 GHz 64-bit quad-core processor and 1GB memory.

The operating system running on the above platforms is Ubuntu 18.04 with Linux kernel 4.14. We used Dlib, a C++ library that provides implementations for a wide range of machine learning algorithms. We chose the Dlib library because it is written in C/C++, and can be easily and natively used in embedded devices.

Why do we need training on edge? #

Cloud is not trustworthy anymore. More and more facts support that breaches on the cloud happen more frequently than before. Nowadays, with more generated personal sensitive data being uploaded to the cloud center, tech companies know better someone than the user. Researchers, no matter whether in the industry or academia, are working in a way where learning from users’ data but also keeping raw sensitive data under users’ control. Many publications have already shown the feasibility of only sharing the after-trained model instead of raw data. One recent popular study on this is Google’s federated learning. During investigating this problem, we found that letting end-user train their data is safe, but sacrifice efficiency. Since one end device has limited resources, training time and power consumption can be disappointing. We believe there must be leverage between privacy and efficiency in some target scenarios.

Fortunately, we observed that users who belongs to the same campus, plant, firm, and community always share similar interests.

Therefore, these co-located users have similar demands in using AI-involved routines. Also, co-located users are easily targeted by the same type of threats, such as ransomware to financial practitioners. Think about this, sending features of a new malware app to cloud services to train neural networks used by antivirus programs. This process may take a long time and a small number of samples may not be recognized by the global neural networks model. A customized local model trained and deployed on the edge can successfully counter the problem. With edge training as a supplement to the cloud, training can achieve better response time and make the whole system more flexible.

Why training on edge is hard? #

Since all co-located users’ devices can be used for edge training, issues and challenges occur when deploying this distributed system. The first challenge is struggling workers. Training devices are heterogeneous, from limited IoT cameras to high-end media centers with powerful GPUs.They are not designed to do machine learning. So, a good edge-based distributed learning framework must be able to handle a variety of speeds in training tasks.

The second challenge is how to scale up clusters. On a campus, thousands and more devices may contribute computing resources to the same training tasks. However, these devices may be located far in physical or network topology. The question of how can we well use them well, without struggling with endless transmission time remains a challenge.

The third issue is the frequent joining and exiting of devices. We can’t rely on each device to faithfully work on training tasks rather than their original workload. Smartly scheduled work balance and handle join/exit issues also need under considered.

Our proposal #

Dynamic training data distribution and runtime profiler

We design a dynamic training data distribution mechanism that helps both the first and the third challenges. Preprocessing data can be transmitted without leakage of raw and sensitive information. This can help struggling workers who can train small batches in order to upload parameters with a similar training time. Also, for extremely slow devices, join and exit of devices cases, dynamic data distribution, and profiler can help with keeping global training parameters from pollution and staleness.

To counter heterogeneity, more approaches were applied in our later research. More details were introduced to the runtime profiler in the later works.

Asynchronous and synchronous aggregation enabled

In our findings, asynchronous and synchronous parameter update have their pros and cons. Keeping sync all the time leads to struggling worker issues unsolvable. However, async’s harm to accuracy and convergence time also needs attention. To carefully choose between these two update policies at the runtime is what we proposed to make use of their own advantages.

Leader role splitting

The idea is to let worker devices with higher bandwidth take leader roles during training. Parameter updating does not require much computation but only needs a great of bandwidth. Devices with sufficient bandwidth can also work as virtual leader devices. This approach helps minimize the physical devices we use and more leaders can further scale up workers’ limits.

Don’t understand the language is not a big deal. If your written language is based on the Latin alphabet(or other languages have space between words), skip the section tokenization.

Why I want to have a Word Cloud #

Recently, I set up a web-based RSS client for retrieving and organizing everyday news. I used TinyTinyRSS, or TT-Rss, a popular RSS client that is friendly to docker. Thanks to developer HenryQW, a well-written Nginx-based docker configuration is already available in the docker hub. With more feeds being added, I found some feeds do not need to be checked daily. Thus I was thinking of creating a script to automatically list all keywords that appear in a last period and generate a heat map kind of figure of it.

Before you go further, I’ll tell you all my settings to give readers a general overview.

My first step is to read all text-based information from TTRSS’s PostgreSQL database. With information, I used a Chinese-NLP library, Jieba, to extract all keywords with the frequency of their occurrence. By using WordCloud, a Python library, a word cloud figure is generated and presented. More details will be discussed in later sections.

Keyword readiness: Get RSS feeds’ text #

My first thought is generating a keyword heat map for economic news of the last week. Since this blog post is more skewed to Chinese tokenization and draws the word cloud figure, I will leave my code here just in case. The SQL connector I used is psycopg2, an easy-to-use PostgreSQL library.

def __init__(self):
    self.dbe = psycopg2.connect(
        host=DB_HOST, port=DB_PORT, database=DB_NAME, user=DB_USER, password=DB_PASS)

def get_1w_of_feed_byid(self, id=1) -> list:
    cur = self.dbe.cursor()
    cur.execute('SELECT content FROM public.ttrss_entries \
        where date_updated > now() - interval \'1 week\' AND id in ( \
        select int_id from DB_TABLE_NAME \
        where feed_id=' + str(id) + ' \
        ) \
        ORDER BY id ASC '
        )
    rows = cur.fetchall()
    return rows

Most arguments are intuitive and easy to understand. The only exception is the argument of the function get_1w_of_feed_byid. This id is the feed index of my subscriptions.

Tokenize with frequency #

Two popular tokenization libraries were used, and I chose Jiebaafter a few comparisons. Before cutting the sentence, we first need to remove all punctuation marks.

def remove_biaodian(text: str) -> str:
    punct = set(u''':!),.:;?]}¢'"、。〉》」』】〕〗〞︰︱︳﹐､﹒
                ﹔﹕﹖﹗﹚﹜﹞！），．：；？｜｝︴︶︸︺︼︾﹀﹂﹄﹏､～￠
                々‖•·ˇˉ―--′’”([{£¥'"‵〈《「『【〔〖（［｛￡￥〝︵︷︹︻
                ︽︿﹁﹃﹙﹛﹝（｛“‘-—_…''')
    ret = ""
    for x in text:
        if x in punct:
            ret += ''
        else:
            ret += x
    return ret

After we have an all-character string, we can call Jieba. By using the function jieba.posseg.cut with or without a paddle, we can have a word list and their “part of speech”. As you can see in the following code, I also did two more works.

First, in the if statement, I only kept all nouns with some categories. Category abbreviations such as “nr” and “ns” represent different “parts of speech”, I attached the categories I used in the following table. For more details, you can find this link.

The second work is only keeping words with lengths longer than 2 characters. In Chinese, there’s no space between words such as in Latin writing systems. Since then, some single-character words such as conjunction words are easy to be misrecognized as specialty nouns. This misrecognition will cause more single-character being regarded as specialty nouns. I am not able to improve the NLP method, so I used an easy way to fix this by removing any words less than 2 characters.

import jieba.posseg as pseg

def get_noun_jieba(self, content: str) -> list:
    content = remove_biaodian(content)
    words = pseg.cut(content)   # Invoking jieba.posseg.cut function 

    ret = []
    for word, flag in words:
        # print(word, flag)
        if flag in ['nr', 'ns', 'nt', 'nw', 'nz', 'PER', 'ORG', 'x']:   # LOC
            ret.append(word)
    return [remove_biaodian(i) for i in ret if i.strip() != "" and len(remove_biaodian(i.strip())) >= 2]

• Word category names and abbreviations

AbbreviationCategory name/ Part of speechnrPeople name nounnsLocation name nounntOrganization name nounnwArts work nounnzOther nounPERPeople name nounORGLocation name nounxNon-morpheme word

With all words extracted, we can easily calculate their frequencies. After this, we can using the following line of code to print a sorted result to verify correctness.

noun = seg.get_noun_jieba(test_content)
# ... Calculate frequency of above word list ...
print(sorted(a_dict.items(), key=lambda x: x[1]))

Draw word cloud #

With a keyword and frequency dictionary(data structure), we can just call built-in functions from Wordcloud library to generate the figure.

First, we need to initialize an instance of wordcloud class. As you can see in my code, I set it with 6 parameters. Width and Height of the canvas, the maximum amount of words used to generate the figure, the font of words, the background color, and the margin between any two words.

After having the instance, we call the function generate_from_frequencies and pass the keyword dictionary to it. The return value of this function is a bitmap image, which we can use Matplotlibto plot it to your screen.

I tested my plot on ubuntu-subsystem on Windows 10, unfortunately, Matplotlib under subsystem depends on the x11 window manager, and it’s not default available on Windows. We need to install an x11 manager to support this. Xming is the one I used.

from wordcloud import WordCloud
import matplotlib.pyplot as plt

font_path = "./font/haipai.ttf"
output_path = "./font/out.png"

def show_figure_with_frequency(keywords: dict):
    wc = WordCloud(width=828, height=1792, max_words=200, font_path=font_path,
                   background_color="white", margin=1).generate_from_frequencies(keywords)
    plt.imshow(wc)
    plt.axis('off')
    plt.show()

If everything work fine, a word cloud figure will show up in a new window. My version looks like this.

The two largest words in the figure are “新冠” and “新冠病毒”, both meaning “Covid-19” (This figure was in the week of the second covid spur in Beijing, China). The size of the image fits my phone screen and I can use an app to automatically sync it to my phone’s wallpaper. However, in this image, too many location nouns are presented. This will be something I can make progress on in the future.

Xv6 System call #

To invoke a system call, we have to first define a user-mode function to be the interface of the kernel instruction in file user.h.

void function (void);

This interface-like function will then pass the function name, in this case function, to usys.S. When using user mode function in programs, usys.S will generate a reference to SYS_function and push system call number of this function into %eax. After that, system can know from syscall.c and determining whether this system call is available. We must define same name system function and add it into syscall.h and syscall.c.

#define SYS_function ##  // ## is the system call number
[SYS_function]  sys_function // real system function name
extern int sys_function(void); // real system function declaration

After adding these sentences to syscall files, we can implement real function in specific place where you want to make the function works well.

Sometimes, we need to pass variables among system calls. In this case, variables’ values are not necessary and even can’t be pass directly into system_function. When invoke a system call function, all variables of this system call will be pushed into current process’ stack. In file syscall.c, multiple functions are provided to get these variables from the process. I won’t waste time on explaining how to use these functions especially when elegant and detailed comments were written in source codes. However, I will explain concepts and how process organized and works in xv6 in future articles.

Debug xv6 with gdb #

Please make sure that you have used gdb before. If you never used gdb, you may write a simple 50-100 lines c code and practice how to use gdb first.

• GDB Manual

• GDB cheatsheet (pdf)

To make sure xv6 gdb enabled, please check if .gdbinit.tmpl file exist. This file is used for generate .gdbinit file which you can late consider it as a configuration for gdb.

Before running the xv6 instance in QEMU, one more thing you need to know is that using gdb to debug xv6 must be attached remotely. This is because xv6 was running within QEMU, and emulator is virtually gapped from the host device. Later when you start debugging, QEMU will open a gdb server to let gdb client connect to.

Once you want to start, use following command to compile and run xv6

$ make qemu-nox-gdb
*** Now run 'gdb'.
qemu-system-i386 -nographic -drive file=fs.img,index=1,media=disk,format=raw -drive file=xv6.img,index=0,media=disk,format=raw -smp 2 7

At this moment, it feels xv6 was stuck, this is because QEMU is ready to be connected by the gdb client. You may use the .gdbinit to automatically finish this remote connection by simply typing in the following command in another terminal.

$ gdb -x .gdbinit
GNU gdb (Debian 8.2.1-2+b3) 8.2.1

...

The target architecture is assumed to be i8086
[f000:fff0]    0xffff0: ljmp   $0x3630,$0xf000e05b
0x0000fff0 in ?? ()
+ symbol-file kernel
warning: A handler for the OS ABI "GNU/Linux" is not built into this configuration
of GDB.  Attempting to continue with the default i8086 settings.

(gdb)

Now within this gdb client shell, type ‘c’ to continue the xv6, and you will see xv6 start execution in the first terminal.

At this moment, you may add breakpoints to your code to see if your code is correctly implemented or not.

One more thing, if you open .gdbinit file, you’ll find that it by default connect to a localhost target. If you are working on some other environment where the target and the client were not placed in the same device, change the localhost to IP address correspondingly. Using ssh may connect to different physical devices under the same domain name, this is because load balancers were used. To check the IP address, search command IP.

target remote localhost:28467
# target remote [ip-addr]:28467

Further readings #

Post regarding how Xv6 scheduler works from kernel bootstrap to first user process

2016-10 #

Time Experiment of rsync #

Patch is based on rsync with version 3.1.2. [Rsync|Patch]

How to collect data #

Basically, everything of transmission time and computation time will be output with overall time will be printed on the console. But we also need some bash script to collect data through different size of random size and with different modification through them.

• Start from 8K to 64M, modify at beginning, [Bash script]

• Start from 8K to 64M, modify at last, [Bash script]

• Start from 8K to 64M, modify at random place with a (slow) python script, [Bash script|Python program]

Time Experiment of seafile #

Patch is based on seafile 5.1.4. You can find the release from seafile official repo. You may follow official compile instructions from here. [Patch no longer avaiable, new version at following sections]

How to collect data #

We also need everything be done using scripting. But this time I only design added some distance between two increasing files’ sizes.

• Start from 8K to 16M, 4 times increasing, modify at beginning/ at 1024 different places with python script. [Bash Script|Python program]

• After using this auto testing script, everything of output will be marked in log files of seafile, which located in ~/.ccnet/log/seafile.log

• We need to use this simple awk code and vim operation to extract data.

# CDC: content defined chucks
# HUT: Http upload traffic
# ALL: overall time of one commit & upload
awk '/CDC|HUT|ALL/ {print $4,$5}' ~/.ccnet/log/seafile.log > results.stat

Install Seafile on odroid xu #

Due to failure of my cross-compile to seafile on android. I used develop board as a replacement experiment platform for ARM-seafile testing. I used a odroid xu as hardware standard. Because all I need is an ARM platform, only an ARM-Ubuntu is enough for me. But develop prototype on a board is much fun than coding, I won’t address much this time. But I’ll start a blog telling some really cool stuff I made for a strange aim.

To install a ubuntu with GUI is my all preparation work. I found to way to do this.

armhf is a website for arm-based ubuntu. It has a detailed instruction to follow at here. They also provide ubuntu 12.04/ 14.04 and debian 7.5 to choose. But unfortunately odroid xu’s hdmi output doesn’t supported by ubuntu native firmware. So install ubuntu-desktop might can’t be boot up for video output.

Burn images is much easy to install a pre-complied ubuntu system. I found this on odroid xu’s forum, which contains xubuntu image [download] for odroid xu. With this image, you just need to use dd command to write whole system mirror into sdcard.

# If .img end with xz, use this command to uncompress first
unxz ubuntu-14.04lts-xubuntu-odroid-xu-20140714.img.xz    
# Burn image into SD-card
sudo dd if=ubuntu-14.04lts-xubuntu-odroid-xu-20140714.img of=/dev/sdb bs=1M conv=fsync
sync

2016-11 #

Android Kernel #

How to build an Android Kernel? #

Generally, I won’t tell anything in this parts, just mark some related links, and point out some mistakes or error solutions.

Google Official Guide

If you don’t have AOSP sources, you have to download prebuilt toolchains which recommended in this guide might not be correct. Use following links to choose your fitting tools.

ASOP git root, under sub class “/platform/prebuilts/gcc”

Packing and Flashing a Boot.img [highly recommend]

2016-12 #

Android Kernel #

How to compile with ftrace? #

If we want to debug under android, ftrace is a great tool for working. But, ftrace is not available in android if we used default configure file. Android kernel configuration is in arch/arm64/kernel/configs. We need to add few lines under that.

CONFIG_STRICT_MEMORY_RWX=y
CONFIG_FUNCTION_TRACER=y
CONFIG_FUNCTION_GRAPH_TRACER=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_PERSISTENT_TRACER=y
CONFIG_IRQSOFF_TRACER=y
CONFIG_PREEMPT_TRACER=y
CONFIG_SCHED_TRACER=y
CONFIG_STACK_TRACER=y

How to extract android images: Dump an image #

If we want to hold a rooted status after flashing boot, we need to extract an image from android devices. We can first use following command to find which blocks belongs to. According to some references, this article provide three ways to dump an image, I picked one for easy using.

adb shell
ls -al /dev/block/platform/$SOME\_DEVICE../../by-name # {Partitions} -> {Device Block}

# dump file
su
dd if=/dev/block/mmcblk0p37 of=/sdcard/boot.img

My current solution is using AP to forward all SSL traffic to a proxy, charles proxy is my first choice (Prof asked). It’s a non-free software which still update new versions now. So mainly, I’ll talk about how to charles SSL proxy.

Preparations #

• Monitor device situation: Linux Machine with wireless adapter

• Download the newest version(4.0.1) of charles

• Target android devices with root privilege

Install Charles and Configuration #

• You have to install charles first. After downloading the charles proxy, you have to unzip it and configure some basic settings.

# open charles first
./bin/charles

• Save charles’ private key and public key

In Help -> SSL Proxying -> Export Charles Root Certificate and Private Key, enter a password and save the public and private key in *.p12 format. You also need to save charles Root Certificate, it also contains in the same menu. For convience, save it as *.pem format.

• Set Proxy and SSL Proxy