Author Archives: Hong

About Hong

The maintainer of topbug.net.

View the TLS Certificate Details of a Website on the Command Line Using GnuTLS

Last updated on December 27, 2023

With GnuTLS, we can view the certificate details of a website with the following commands (replace "example.com" with the website of your interest):

gnutls-cli --print-cert example.com < /dev/null | certtool --certificate-info

In the command above, gnutls-cli --print-cert example.com < /dev/null prints the certificate of the website in PEM format to the standard output. Its output is then sent as the standard input to certtool --certificate-info, which prints information on the given certificate.

Continue reading

Platform Dependent Python Coverage Test with Tox

Last updated on December 20, 2020

When testing Python programs, coverage.py is often used in measuring code coverage, and enforcing 100% code coverage is regarded as a good practice:

# .coveragerc
[coverage:report]
# Enforce 100% coverage test
fail_under = 100
show_missing = True

However, if there are some lines of code that are platform dependent (i.e., they are never executed on at least one platform), code coverage tests usually fail. For example, the following code snippet would always lead to a coverage that is less than 100% on a platform other than Windows:

if os.name != 'nt':
    # Do something if the OS is not Windows...

You can ask coverage.py to ignore this block by adding a comment # pragma: no cover, but then coverage.py would ignore it on all platforms, including all non-Windows platforms. If you use tox for testing, this issue can be resolved cleanly.

Continue reading

Catching FileNotFoundError? Watch Out!

Last updated on October 3, 2020

In Python, FileNotFoundError is an exception that is raised when a requested file does not exist. Many people assume that when their programs fail to open a file in read-only mode or delete a file, FileNotFoundError must be raised and they should only need to process that. For example, some people would write code similar to:

def process_file(path):
    import sys

    try:
        f = open(path, 'r')  # or os.remove(path)
    except FileNotFoundError as e:
        print(f"File {path} not found!", file=sys.stderr)
        return
    # process the file...

However, this code may actually trigger unexpected errors. The reason is that, the failure to open a file in read-only mode or delete a file is not necessarily caused by the non-existence of the file. Very often, it's for different reasons: insufficient permission, or the file is a directory. In this case, PermissionError or IsADirectoryError would be thrown instead of FileNotFoundError. So, in the example above, one would want to catch all of them:

Continue reading

Makefile Template for a Shared Library in C (with Explanations)

Last updated on October 28, 2019

tl;dr: Save the following file as Makefile and change the source files to the ones that you intend.

# Makefile template for a shared library in C
# https://www.topbug.net/blog/2019/10/28/makefile-template-for-a-shared-library-in-c-with-explanations/

CC = gcc  # C compiler
CFLAGS = -fPIC -Wall -Wextra -O2 -g  # C flags
LDFLAGS = -shared   # linking flags
RM = rm -f   # rm command
TARGET_LIB = libtarget.so  # target lib

SRCS = main.c src1.c src2.c  # source files
OBJS = $(SRCS:.c=.o)

.PHONY: all
all: ${TARGET_LIB}

$(TARGET_LIB): $(OBJS)
	$(CC) ${LDFLAGS} -o $@ $^

$(SRCS:.c=.d):%.d:%.c
	$(CC) $(CFLAGS) -MM $< >$@

include $(SRCS:.c=.d)

.PHONY: clean
clean:
	-${RM} ${TARGET_LIB} ${OBJS} $(SRCS:.c=.d)

The above code snippet is also available on GitHub gist.

Continue reading

Pitfalls in Using a Standard “Global” C++ Function: Its C Counterpart May Be Camouflaging

Last updated on September 30, 2019

A large number of C++ functions in the standard library are extended from standard C functions, such as qsort(), memcpy(), etc. Among these functions, many have overloaded their C counterparts. For example, abs() in C++ is overloaded for both integral and floating-point types, while it is only defined for int in C. These functions, however, are often unwittingly misused as global functions and produce unexpected results.

Continue reading

Automatically Conceal Sender’s IP Address in Email Clients via SSH Tunneling

Last updated on May 5, 2021

Desktop email clients, such as Thunderbird and Claws Mail, are preferred over their web counterparts by many professionals and power users due to their additional convenience and security. However, one big downside is that they often expose the sender's IP address to the receivers, since many SMTP servers record the sender's IP address and put it in the header, something similar to Received: from [xxx.xxx.xxx.xxx] (my.example.com. [xxx.xxx.xxx.xxx]). This, unfortunately, puts the sender's privacy in great jeopardy, as an IP address can reveal so much information including location, ISP, and institution names.

To address this issue, one simple solution is to let the email client connect via a proxy. While a system-widely available proxy works for many users, some of us just want our email clients, but not other programs, to go through a specific proxy. In this post, I'll demonstrate how to use an email client automatically via SSH tunneling. The instructions are specifically tailored for GNU/Linux and MacOS users, as it involves some uses of UNIX commands and bash scripts; if you are on Windows, you can still follow the instructions with the help of Cygwin.

Continue reading

Nikola: How to Deploy Compiled Webpages to a Different Git Repository

Last updated on November 18, 2018

Nikola is one of the most popular static website generators. It compiles source files into final publishable webpages offline and then uploads those files to a web host. Compared to dynamic websites such as those powered by PHP or Ruby on Rails, static websites offer better security and faster page loading.

Nikola provides some utilities to ease the deployment procedure (i.e., uploading compiled webpages), especially for deploying as GitHub pages. Unfortunately, Nikola does not (and its team does not plan to) provide a direct way to deploy the compiled webpages to a git repository that is different from the one that hosts the source files. This is often useful when you want to hide the source files in a private git repository and leave the git repository that hosts the compiled webpages public. Luckily, Nikola provides customizable deploying commands. Assuming output is the directory where the compiled webpages are located, change the value of DEPLOY_COMMANDS using the following in conf.py (replace me@example.com with your email address, https://xuhdev@github.com/xuhdev/xuhdev.github.io.git with your designated git repository on GitHub/GitLab/BitBucket/etc., and master with your designated branch):

DEPLOY_COMMANDS = {
    'default': [
        "cd output && git init && git config user.email me@example.com && touch .nojekyll && git add .",
        "cd output && git commit -a -m 'Nikola'",
        "cd output && git push -f https://xuhdev@github.com/xuhdev/xuhdev.github.io.git master",
    ]
}

Now running nikola deploy should deploy the compiled webpages to your designated git repository and branch.

Technology is not Everything: Non-Technical Aspects to Consider for Open Source Projects

Last updated on November 21, 2018

Open source software, also known as free and open source software (FOSS), free, libre, and open source software (FLOSS), and free software, has become more and more popular these days. When starting a new open source software project, we developers tend to think mostly on the technical side of this project, e.g., what programming languages and frameworks to use, how to design the architecture of the software, what platforms to target at, etc. We not only think, we actually think about these really carefully: We look around for advice, struggle in our mind, and, sometimes, even after years, we still argue that we should rewrite the software in a different programming language. On the other hand, we often take the non-technical side too light without much thinking: We use a specific source code hosting service simply because everyone else uses it, we use a specific license simply because this is the only license that we are able to read through once and (vaguely) understand, etc. These decisions, however, will heavily affect the style, the advancement,or even the survival of our open source software projects.

This post aims at draw developers' attention to non-technical aspects of open source software projects. We will have a brief overview of some non-technical aspects that can be important in open source software projects (while more aspects and details can be further discussed in the future).

Continue reading

Swap Training and Test Data During Cross-Validation in scikit-learn

Last updated on October 11, 2018

Scikit-learn is a well known Python machine learning library. It provides various utilities for machine learning, including those for cross-validation. In a standard \(K\)-fold cross-validation, the data are split into \(K\) subsets (with equal size). There are \(K\) rounds of training and testing. In each round, one subset is used as test data and all other subsets are used as training data. Under this setup, as long as \(K > 2\), there are always more training data than test data in each round of the cross-validation. Whilst this is desirable in most cases, in some machine learning applications, it is more desirable to have training data less than test data. For example, in graph embedding, each node in the network has a vector representation and labels. When running cross-validation, it is more desirable to use a smaller number of nodes as training data than the number of nodes as test data, since this better mimics the real-world scenario in terms of the amount of available training data (e.g., here). In scikit-learn, we can achieve this by swapping training and test data.

Continue reading

Creating Multiple-Choice Exams with Answering Boxes Using LaTeX

Last updated on December 26, 2017

For a multiple-choice exam, to ease the grading procedure, it is often preferred to ask students to write their answers collectively in an answer sheet with their choices of answers filled in boxes. However, LaTeX, in particular the exam document class, does not directly provide the feature to automatically generate such boxes. In this post, we will let LaTeX to automatically generate these answering boxes, and with correct answers filled in when the answers document class option is turned on. The effects are displayed below, with correct answers shown and not shown, respectively. Their respective PDF files are also available: Without answers; with answers.

Continue reading