Yoav Goldberg

wdym by "enable manipulation"? the metaphors i think are only "nice to have" are the "projections between spaces" kind of things.

also, there ARE benefits to taking algebra in undergrad. you learn stuff, and the notions of proofs and abstraction are important to acquire. it just not central to DL.

and there is also this:

bsky.app/profile/yoav...

currently it is pre-req also here. but ideally there will be removed and replaced with a more suitable class.

the algebraic terminology is here to stay unfortunately, and we should use it. it doesnt mean we DL is "built on" linear algebra, nor that a linear algebra class should be pre-req.

if you REALLY want to understand DL, you should start by honing your Category Theory skills, as almost everything in DL at its core can be mapped to a functor or an endofunctor.

OR you could do what people actually do in ML these days and associate each symbolic token with a random list of numbers, and let the optimization take care of it.

ah, likely. idk

("any" is a bit extreme because the purists will come and say "ohh but you use commutativity and associativity of addition! thats group theory!!". but i agree with you of course)

it seems to me that a large chunk of ML can actually be characterized as doing dim reduction numerically

ok, but where is PCA important as a building block in ML?

idk the sociology of it. tensors are popular because hardware/software support them, and its convenient to implement batches this way i guess.

where is it used that is central / important?

btw, re the hardware example: the hatdware is not built to perform "norm". it is built to approximate norm over a floating point representation of real numbers. this is highly specific and doesnt enjoy the generality of algebra at all. i think it may not even be commutative and associative.

bsky.app/profile/yoav...

taking it a step further, I'd say in many cases using the algebra jargon is harmful to understanding, and its better to just describe whats really going on. ie, "we add an L2 penalty term" --> want the sum of squares to be small. "project to vocab space" --> compute similarity to each vocab item.

(and the useful parts of the metaphor is also much more geometric than algebric, imo)

i think these are nice metaphors, and i use them daily. i am not at all convinced that they are essential, except for very few select places.

because when we use a term such as "a norm" we get the definition, which is nice, but also a bunch of properties that hold for items of this kind. and if we dont actually rely on these properties later on, then its a waste in terms of what we communicated.

so i would argue that efficiency of computation is nice but also somewhat accidental and not that important. but lets focus on efficiency of communication: my point is kinda exactly around this. in a field that is actually built around algebra, the communication efficiency would be MUCH higher.

טוב גם הטענה "משין לרנינג בנויה על תורת הקבוצות" היא הזויה בעיני. אנחנו כנראה לא מסכימים על ההגדרה של "בנויה על".

אם הטענה היא שאין כלום במתמטיקה בלי אלגברה אז אוקיי, אבל זה קצת טיעון ריק בעיני

איפה אנחנו משתמשים בקונספט הזה? (ואנחנו קצת דוחקים פה את ההגדרה של אלגברה לדעתי, כי אני גם יכול להגיד כך גם שכל אנליזה פונקציונאלית זה בעצם אלגברה, אבל בוא נזניח את זה לרגע)

תרחיב על לפרמל ייצוג וקירוב? למה זה מעניין אותנו ומה האספקטים האלגבריים שם?

what efficiency are we discussing here? efficiency of communication, or efficiency of computation?

סליחה התכוונתי '' "משין לרנינג על אופטימיזציה" זו טענה שאני בכיף ''

"משין לרנינג בנוי על אופטימיזציה" זו טענה שאני מקבל בכייף

i agree that the language is being used. but the language is pretty much the only part that is being meaningfully used.

linear algebra is also concerned with solving systems of linear equations, the representations of linear equations as matrices/vectors, and related objects like spans and bases.

for me, the "algebra" part is the realization that the real numbers and addition/multiplication over them are just a special case of a "group" or a "field", and that many other kinds of groups and fields exist, and can be manipulated similarly, and share many properties.