research on Shiryu NAKANO

Mon, 01 Jan 0001 00:00:00 +0000

ref

Org-mode Test

Mon, 01 Jan 0001 00:00:00 +0000

Overview

研究でも何回も出てくる資料とか読み直してる資料のリンクや、理解した内容をまとめる場所

Math Test

Inline math: $E = mc^2$

Display math:

$$ \int_0^\infty e^{-x^2} dx = \frac{\sqrt{\pi}}{2} $$

Code Block

1
2


def hello():
 print("Hello from org-mode!")

List

Item 1
Item 2
- Nested item
Item 3

[メンタル的なところ](posts/research/attitude)

Org-mode Test

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This is a test post written in org-mode.

Math Test

Inline math: $E = mc^2$

Display math:

$$ \int_0^\infty e^{-x^2} dx = \frac{\sqrt{\pi}}{2} $$

Code Block

1
2


def hello():
 print("Hello from org-mode!")

List

Item 1
Item 2
- Nested item
Item 3

Paper To Read

Mon, 01 Jan 0001 00:00:00 +0000

DSB

SLT for Deep Learning

You're Measuring Model Complexity Wrong — LessWrong
[2410.02984] Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient
[2511.04564] Uncertainties in Physics-informed Inverse Problems: The Hidden Risk in Scientific AI
[2406.10234] Review and Prospect of Algebraic Research in Equivalent Framework between Statistical Mechanics and Machine Learning Theory
Computational Complexity of Learning Neural Networks: Smoothness and Degeneracy

PINNs

Mon, 01 Jan 0001 00:00:00 +0000

Overview

かけるほどわかってない

ref

PINNsの元の論文
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- ローカルにあるので優先的に読む /Users/nkn4ryu/Downloads/1-s2.0-S0021999118307125-main.pdf
[2602.11097] Statistical Learning Analysis of Physics-Informed Neural Networks
[2211.08064] Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications
An efficient wavelet-based physics-informed neural network for multiscale problems - ScienceDirect
[Literature Review] Advancing Generalization in PINNs through Latent-Space Representations
[1711.10561] Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

Scaling Laws in Deep Learning

Mon, 01 Jan 0001 00:00:00 +0000

Overview#

Scaling laws describe how model performance changes with compute, data, and parameters.

Key Papers#

${p(s)}$

${\sigma}$

Kaplan et al. (2020): Scaling Laws for Neural Language Models
Hoffmann et al. (2022): Chinchilla - Training Compute-Optimal LLMs

Observations#

Loss scales as power law with model size, data size, and compute
Optimal allocation of compute between model size and data

Singular Learning Theory (SLT)

Mon, 01 Jan 0001 00:00:00 +0000

What is SLT?#

Singular Learning Theory (SLT) is a mathematical framework developed by Sumio Watanabe for analyzing statistical learning when the model is singular (non-regular).

Key Concepts#

RLCT (Real Log Canonical Threshold): A key quantity that determines generalization error
Free Energy: Measures model complexity in Bayesian inference
Singular Models: Models where the Fisher information matrix is degenerate

References#

Watanabe, S. (2009). Algebraic Geometry and Statistical Learning Theory
Watanabe, S. (2018). Mathematical Theory of Bayesian Statistics

Tips for Attitude

Mon, 01 Jan 0001 00:00:00 +0000

このページは随時更新・追記されて行きます．

研究や勉強・仕事について特に心構えで助けてもらったことのある記事書籍を記録していく．

数学系

研究関連#

君たちはどう研究するか - ｼﾞｮｲｼﾞｮｲｼﾞｮｲ
研究の進め方ランダムネスとの付き合い方について - Speaker Deck
- バックアップ
機械学習の研究者を目指す人へ
論文読みの日課について - ｼﾞｮｲｼﾞｮｲｼﾞｮｲ
- Paper Search from Venues
Awesome CS-Ja PhD Life
- 博士課程をサバイブするために参考になりそう/モチベになりそうな記事や資料のリスト。日本語で読めるCS関連のものが中心です。卒修論に役立つものも掲載しますし、その後のキャリアの話も掲載します。
- この記事にあるmdを一つのnotion DBか何かにして，全て読み終わるまでランダムでリコメンドしてもいいかもしれない．
https://psearch.joisino.net/
- Paper Search from Venues
- 有名なやつ（古典）から読む

事前準備#

勉強は好きなので，趣味としていろんな本を読みたい．