谈体系结构的进步对网络的影响

最近量子位又发了一篇体系结构的进步,TCAM,所谓的三态内容寻址储存器。可以说,从图灵机的角度来说,上层建筑下的基层还有很多没有解决,从现在那么多Startup 在真正的做业务Oriented 的数据库及网络链路优化,体系结构还有很多可以探索的部分。同时,新的架构真的是否安全,如TPU的数据通路是否有没有被侦测到的部分可以被攻击。多数的攻击来自于软硬结合,汇集了多少工程师的智慧结晶。

交换机的简化结构


这是一个去掉2个要素的冯诺伊曼体系结构图,交换机的OutBound 和Input的Throughput是显见的bottleneck,除此之外还有延时,这就需要主存储器性能或者包的传输协议的革新。

三态内容寻址储存器(TCAM)

我在当年写VB的时候记得有个slide的参数,是一个三进制数来表示不动,向上滑和向下滑的参数。而这种0、-1,1三进制在苏联当年的计算设备上有所尝试,可惜最终失败了。0.5或许是更好的一种表示中间态或者亚稳态的编码方式,可以用于模糊匹配,或者Not Set。

CAM本质上是一个数据查找硬件方法,读写数据的速度与RAM相同,查找数据能相对模糊的匹配到数据。

这时ARP 协议从报头或者CRC来验证数据正确性起到了很大的作用,就是不管怎样,数据到了,不管对不对,以最快的速度发出去,等到了再做检验的思路是一样的。(有点像高频交易架构的gateway。

Reference

  1. Constant-time Alteration Ternary CAM with Scalable In-Memory Architecture
  2. 三态内容寻址存储器(TCAM)工作原理

每当有所沮丧的时候

每当有所沮丧的时候

有人说,一个人的全部来自于你现在遇上的他,以及他的所有记忆。

最近我发觉,作为一个正常的大学生,学习能力不会差太多,给定时间学习,总归能超越。比我聪明又比我努力的人多了去了。现在还能坐在这全靠我的幸运。我的弱点过多了,多到很多事情都无法弥补,在大学的时候还有很好的参照系,就是你拿到的成绩在别人之下。可是从小到大我去问别人的时候,总能发现他们其实并解释不清楚很多事情,只是纸面上不知怎的呈现出来了,在风雨飘摇的拿出了那点成果。我把其归结为我的研判失误。

对远的事物看的很透彻,对近在眼前的事毫无关心,甚至有点因噎废食。

为什么人要这么卷

我不是一个对眼前利益把握很深的人,或者说一直让自己处于舒适区以外,可能跟小时候一直和大朋友玩,即便被鄙视。实际上我一直是被鄙夷的状态。社交的不足让我更对同龄人关心的无所适从。我超前的干了很多不属于我这个年龄干的事情,也选择拒绝了很多不必要的事情。我的能力从来都是在量变到质变的过程中。我发现了学校对我的揠苗助长,同时也发现了同学真的很有水平。

我其实一点也不卷,我从来,其实有点在意分数,挺在意别人对我的评价,胜过知识点本身,都大三了,还什么都不会,也就写写简单的可能数十年前的大家就广为讨论的算法与系统结构,想想并发带来的优劣,除此之外,别无他毕生的力气。

我的改变

加油弄懂每一个知识点吧,SC20 加油!

[Network] 网络链路层路由算法总结

Routing protocols

  • Routing information protocols(RIP)
    • Algorithm: Distance Vector
  • Open Shortest Path First (OSPF)
    • Algorithm: Link State
  • BorderGatewayProtocol (BGP)
    • Another type of vector Routing: widely used in the AS Protocals
    • demo
    • Problem with integrate wit intra domain Routing
      • static method ALL unknown IPs send to D.
      • Entry Translation: high cost
      • interior BGP

ARP

之前打ctf 的时候搞过这个协议,主要是在MAC 层能对网卡欺骗,从而把destination 当成自己,就能抓取同路由器下的其他设备的包。

这种需要规避就得ip-mac 绑定。

bilibili总部的见闻——中年油腻大叔的青年回忆录

主要是那天想吃肉了,也想一个人出去遛哒。回了趟杨浦区高中,骑去B站门口看小姐姐下班,五角场的二次元浓度也很高,最后在高中时代就很喜欢去溜达的复旦江湾和合生汇。

高中没什么可以回忆的,全是被同学草虐的回忆。

倒是我想对我自身的成才路径画上了个问号,即在智商严重缺失的情况下如何反败为胜。我是个反应和持久力都还不错的人,觉得学习上最大的问题是极度依赖记忆,而非认真的推导。

[Network] 几个Mac层的协议

我校网络老师不太Care 概率模型下的网络分析,只Care 实现。(但是我最近正好在学概率统计,权当一道作业题复习

ALOHA协议

主要思路就是让所有能发的人都发,有错误就随机掷骰子决定发送,如果碰撞,随机范围翻倍再掷骰子。


可问题是,非常容易冲突。

如果我们做一道概率题

  1. 帧时T:发送一个标准长的帧所需的时间
  2. 吞吐率S:在一个帧时T内发送成功的平均帧数(0<S<1,S=1时信道利用率100%)
  3. 运载负载G:一个帧时T内所有通信站总共发送的帧平均值(包括原发和重发帧)(G≥S,G=S表示无冲突)
  4. P0:一帧发送成功(未发生冲突)的概率,发送成功的分组在已发送分组的总数中所占的比例;公式:S = G*P0

两个标准长的帧才会第二次碰撞,舍冲突危险期为2T,同时设这是的帧平均值为2G一个T内生成k个帧到的概率符泊松分布。

由柏松分布可知:

\(\operatorname{Pr}[\mathrm{k}]=\frac{\mathrm{G}^{\mathrm{k}} \mathrm{e}^{-\mathrm{G}} }{ \mathrm{k} !}\)

\(P(\text{success in 2T})=Pr(0)\times Pr(0)=e^{-2G}\)

带入S= G*P0 得
\(\mathrm{S}=\mathrm{Ge}^{-2 \mathrm{G}}\)

最高信道利用率是18.4%

Time-slotted ALOHA

  1. 分隙ALOHA是把时间分成时隙(时间片),时隙的长度对应一帧的传输时间
  2. 新帧的产生是随机的,但分隙ALOHA不允许随机发送,凡帧的发送必须在时隙的起点
  3. 冲突只发生在时隙的起点,冲突发生时只浪费一个时隙,一旦某个站占用时隙并发送成功,则在该时隙内不会出现冲突

显然刚刚 \(P(\text{success in 2T})=Pr(0)=e^{-G}\), \(S=\mathrm{Ge}^{-2 \mathrm{G}}\)
最高信道利用率是36.8%

CSMA/CD

一种带有冲突检测的载波监听多路访问,可以检测Mac传输的冲突。

主要流程是

  • 以广播发,看看有无其他节点(carrier sense)没有其他包就发包。
  • 检测 carrier detection。 如果碰撞再广播碰撞了。然后掷骰子重新发包
  • 15次失败报告timeout

CSMA/CA

WLAN 中实现不了CSMA/CD 主要原因是有hidden和exposed 的情况。一种不太好的解决方法是RTS-CTS。可这比较容易被攻击。

组里的几个邀请

上周、上上上周和上上上上周,我们迎来了组里的讲座,一个是新加坡国立做Software Analysis的Prof Visit。一个是用DTMC来解释音频的项目。The author XIAONING DU is now in Sydney Tech。另一个是在交大读的研究生,NTU 读的两面本和博。 The author Hongxu Chen

我对前者的直观感受就是,可能这个方向的博士不是很难拿吧,但deepstellar当时还是热点,现在有点凉,但证明能干一些事情了,她趁火拿了好几篇adv和benign。我对后者的感受,很强,我到博士不一定有其水平的一半。

RNN to DTMC


就是一坨统计堆出来的可解释性。
进一步抽象


这里用的公式\(\operatorname{Dist}\left(\hat{s}, \hat{s}^{\prime}\right)=\Sigma_{d=1}^{k}\left|I^{d}(\hat{s})-I^{d}\left(\hat{s}^{\prime}\right)\right|\)

最终的结果

最后用统计做了一堆相似性的bound “证明”,就说RNN聚类抽象出来的state 和DTMC 一一对应。想法很新颖,但其实回头想没什么内涵。

MUZZ

这里 Thread-aware 就很厉害。

相当于独自开个领域


可我事后问了他scalable的问题,他的回答:

go channel不行 java lock 有几个坑 lock threading 不能很准 java分析 fuzzing oracle 异常 jlang 方舟编译器。 scala native不靠谱 动态 gc llvm 不会做 jvm 抽象等级 z一致性 印度 chenhao 学术界。llvm uiuc 
爷 工业界 fuzzing ok2。

他还有几篇

坚定了我去美国找个工位的目标,加油~

On database of Go and Kubernetes and Rust

A few days during the long vacation in China, I found some stuff good to play. The intensive was to figure out a fast framework for my ugly blog, which turns out to be non-sense. But the process of figuring out them is so so interesting.

My demands

My need is to have gitment embeddings work so that the trash comments will be filtered. Those can be handwritten by myself, but I don’t currently have time to do systematic research on javascript, only applying the latest wheel is enough for me. Besides, I’m in great need of markdown writing experience, so gatsby, Hexo and Hugo are my choices.

Rust

First, I consult on some rust written blog, with which I provide a great amount of efforts. https://github.com/ramsayleung/blog was a fantastic one. I found it utilize diesel to make mapping by struct in rust like:

table! {
    post (id) {
        id -> Int4,
        title -> Varchar,
        subtitle -> Varchar,
        raw_content -> Text,
        rendered_content -> Text,
        create_time -> Timestamp,
        modify_time -> Timestamp,
        post_type -> Int4,
        hit_time -> Int4,
        published -> Bool,
        slug_url -> Varchar,
        enable_comment -> Bool,
        tag -> Jsonb,
    }
}

table! {
    user (id) {
        id -> Int4,
        username -> Varchar,
        hashed_password -> Varchar,
        create_time -> Timestamp,
        modify_time -> Timestamp,
        email -> Varchar,
        avatar_url -> Nullable<Varchar>,
    }
}

table! {
    visitor_log (id) {
        id -> Int4,
        ip -> Inet,
        access_time -> Timestamp,
        user_id -> Int4,
    }
}

allow_tables_to_appear_in_same_query!(
    post,
    user,
    visitor_log,
);

I consult on the database of our schools'. I thought it was pgdb, which I guess right. In terms of the static website for blogging, the database seems laggy and out of date. I eventually found out that even the database API is written in rust, but the speed of calling prosgredb is not that fast, within 10s ms per outer key consultation. The web parts is pure js, but not wasm. The rust part only account for its request logic dealing with backend, still the js to get data from the database, which is sad.

Rust is still not a frontend ready language, although it claims to be a fast and high throughput language in terms of dealing with data. Although they have https://github.com/SASUKE40/yew-starter for wasm but still javascript, so why not just javascript?

nearly all the data storing in the language that utilize API use mapping

For example JSON

HUGO

Hugo is written in go. At the jump, I have some experience of dealing with time serialized data (LSM) of HPC data using go API. go is really an out-of-box language so you don’t care much about the memory leakage and semaphore stuff for multithreading programs. Because many of the company is utilizing the language, there’s a bunch of resources and society for CRUD and business code, from database to HTTP sever, from JSON to YAML. HUGO is just another part of it. I gain much information from the blog there https://draveness.me/few-words-time-management/.

Gatsby

React implementation, React components required. I’m not so familiar with javascript and only had one project with LEAFERX, a nice guy. I eventually turn back to php using wordpress.

Why Rust is not ready and Go is ready.

Inside the choice of blog, I talked about the Rust right now is porting everything out of its good & safe logic of itself. The scheme of the rust deisey is just dumby. Rust is not ready for high throughput program unless it has better package for native web deployment. Go is ready for it has its own coroutine, c++2a is catch up with it later on. But go is combining the java developer to make it has c++ speed with single lines. Like Drogan/Drongan.

http package of go

The net/http language of Go wraps both the HTTP client and server implementations, in order to support better scalability, it introduces the net/http. An interface to the HTTP request, where the caller takes the request as an argument to get a response to the request, and net/http. Handler is mainly used by the HTTP server to respond to client requests.

scheduler of go

Signal-Based Preemptor Dispatcher - 1.14 ~ so far

  1. Enabling signal-based true preemption dispatch.
    Garbage collection triggers preemption scheduling when the stack is scanned.
  2. Not enough time points have been seized to cover the full range of edge cases.
    static void schedule(G *gp) {
    schedlock();
    if(gp != nil) {
        gp->m = nil;
        uint32 v = runtime·xadd(&runtime·sched.atomic, -1<<mcpuShift);
        if(atomic_mcpu(v) > maxgomaxprocs)
            runtime·throw("negative mcpu in scheduler");
        switch(gp->status){
        case Grunning:
            gp->status = Grunnable;
            gput(gp);
            break;
        case ...:
        }
    } else {
        ...
    }
    gp = nextgandunlock();
    gp->status = Grunning;
    m->curg = gp;
    gp->m = m;
    runtime·gogo(&gp->sched, 0);
    }
    

How overlay network is written in go.

Overlay networking is not actually a new technology, it is a computer network built on another network, a form of network virtualization technology that has been facilitated by the evolution of cloud virtualization technology in recent years.

In practice, we typically use Virtual Extensible LAN (VxLAN) to set up an Overlay network. In the following diagram, two physical machines can access each other over a three-layer IP network.

Reference

  1. https://draveness.me/whys-the-design-overlay-network/
  2. Kubernetes 源码剖析

Solve the spack install gromacs+cuda+mpi not compatible with [email protected]

For a temporary solution: you can merely download their upstream release from here that fixed the cuda-11 not compatible issue. and put them to /path/to/spack/var/spack/cache/_source-cache/archive/cd/cd12bd4977e19533c1ea189f1c069913c86238a8a227c5a458140b865e6e3dc5.tar.gz.

The checksum can be gotten by sha256sum gromacs-prepare-2020.4.tar.gz. Besides, you should modify the 2020.4 checksum in /path/to/spack/var/spack/repos/builtin/packages/gromacs/package.py, add version('2020.4', sha256='cd12bd4977e19533c1ea189f1c069913c86238a8a227c5a458140b865e6e3dc5').

Then you can simply run spack install [email protected]+mpi+cuda

live2d imbed to your blog and appear on all end device with your tone

The nutshell is to have a virtual sakurajima mai that can have your AI tone for sure.

some food for thoughts:

  1. Chi-Bot : a nlp implementation that take some of the words from Mr. Chi to corpus and utilize a model to generate more. Some of the original input can be listed below:我好菜啊
    我菜爆了
    我是什么垃圾
    我失败了
  1. live2d : a cross platform frontend for animation creation, in which much animi characters are uploaded by the fans worldwide.
  2. live2dviewerex : a android and steam (windows & MacOS) apps.
  3. live2d-wp : a wordpress plugin by 33.

First, it's easy to get that the words from the live2d is stored as waifu-tips.json.

{
   "waifu": {
       "console_open_msg": ["哈哈,你打开了控制台,是想要看看我的秘密吗?"],
       "copy_message": ["你都复制了些什么呀,转载要记得加上出处哦"],
...
  }

What I only care about is the "mouseover":, and it's easily modified by the pyjson and nlp generated stuff. So just frequently modify them.

mai jiang is a master piece by neko_kk, which can be output by wpk. and just replace them with the 33. Here we go.