Karl Xie

做正确的事情，觉察和接纳恐惧和焦虑，让它们像风一样穿过你的身体。

平静的大海培养不出优秀的水手。

Google Colab

Google Colab,

short for Colaboratory, is a free, cloud-based platform that allows users to write and execute Python code in a Jupyter Notebook environment.

colab.research.google.com

PyTorch documentation

pytorch.org/docs/stable/...

有一些单词天然就更基本，更重要，使用频率更多。

将相近的语言比较着学是一种非常有趣的体验，比如英法德。

单词是描述自我和世界的基本单元，能够在写作和口语中主动使用的单词越多，越精细、微妙，说明你对自我和世界的认知水平越高。

Everyone should read the Constitution of the United States.

悲观没有任何正面意义，只会消耗心力。

宁可错误地乐观，积极地解决问题，也不因为正确的悲观而缺乏行动力，错失未知的机会。

语言的边界就是精确思维的边界。

Understanding LLMs: A Comprehensive Overview from Training to Inference The introduction of ChatGPT has led to a significant increase in the utilization of Large Language Models (LLMs) for addressing downstream tasks. There's an increasing focus on cost-efficient training...

Understanding LLMs: A Comprehensive Overview from Training to Inference

Yiheng Liu, etc.

Source: arxiv.org/abs/2401.02038

Glorot, Bengio. Understanding the difficulty of training deep feedforward neural networks. 2010

Source: proceedings.mlr.press/v9/glorot10a...

Inductive Representation Learning on Large Graphs Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing ...

Inductive Representation Learning on Large Graphs

William L. Hamilton, Rex Ying, Jure Leskovec

Source: arxiv.org/abs/1706.02216

Attention Is All You Need The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and d...

Attention Is All You Need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin

Source: arxiv.org/abs/1706.03762

How Does Batch Normalization Help Optimization? Batch Normalization (BatchNorm) is a widely adopted technique that enables faster and more stable training of deep neural networks (DNNs). Despite its pervasiveness, the exact reasons for BatchNorm's ...

How Does Batch Normalization Help Optimization?

Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, Aleksander Madry

Source: arxiv.org/abs/1805.11604

Group Normalization Batch Normalization (BN) is a milestone technique in the development of deep learning, enabling various networks to train. However, normalizing along the batch dimension introduces problems --- BN's e...

Group Normalization

Yuxin Wu, Kaiming He

Source: arxiv.org/abs/1803.08494

Understanding LSTM Networks

Source: colah.github.io/posts/2015-0...

Andrej Karpathy blog: A Recipe for Training Neural Networks

Source: karpathy.github.io/2019/04/25/r...

AI Notes: Initializing neural networks - deeplearning.ai In this post, we'll explain how to initialize neural network parameters effectively. Initialization can have a significant impact on convergence in training deep neural networks...

Initializing neural networks

Source: www.deeplearning.ai/ai-notes/ini...

PyTorch, Dynamic Computational Graphs and Modular Deep Learning Deep Learning frameworks such as Theano, Caffe, TensorFlow, Torch, MXNet, and CNTK are the workhorses of Deep Learning work. These…

PyTorch, Dynamic Computational Graphs and Modular Deep Learning

source: medium.com/intuitionmac...

番茄钟+费曼学习法+一天至少有半小时充分放松娱乐，这是过去一年经过实践证明效果良好的自律方法。

长期的计划和抽象的目标，一定一定要拆解成每天、每周、每月的具体行动，贯彻落实。

* rapid feedback iteration

5 most important keys to success:

* passion
* persistence
* focus on the core issue
* attention to detail
* PDCA
* calm

In the midst of the uncertainties of life, some feeling that a possible access to protection existed was essential. Men depended on the gods for reassurance in a capricious universe.

Our species has almost certainly not shown any improvement in innate capacity since the Upper Palaeolithic.

音乐方面：明确了自己的爱好，主要是合成器浪潮、现代纯音乐、1980s摇滚，解锁了大量的合成器浪潮歌曲和现代纯音乐。

影视作品方面：看了周处除三害，致命女人一二季，宇宙探索编辑部，里斯本丸沉没，疯狂动物城，因果报应，杀人回忆，禁闭岛，后翼弃兵，狼嚎。

计算机技能方面：重新熟悉了Markdown、Python、Jupter Notebook。

游戏方面：解锁了艾尔登法环和赛博朋克2077。