Skip to content

Latest commit

 

History

History
190 lines (119 loc) · 11.2 KB

notes.md

File metadata and controls

190 lines (119 loc) · 11.2 KB

Chapters 1, 2 and 3

Some threads that we discussed

  • Is there a line to be seen that goes Konrad Lorenz -> Gibson -> Paul Cisek
  • What is Gibson reacting against? What was vision research back then? Those were the days of peak-Movhson, perhaps.
  • Is this book about how animals and environments co-develop? Or is the idea underlying this idea there, but this book specialized to humans?
  • There's a fundamental tension in the book: a generative model of visual information is posited (surfaces, BDRFs), but it's implicitly assumed (perhaps?) that the animal can perform its actions without an internal model.
  • What does 85% of V1 do?
  • Do we even know what attention is (William James: we all know what it is...)
  • There's an underlying thread of compositionality: objects are made of stuff made of things. How does this break down?
  • Kant vs. Hume: obligatory Existential comic
  • Affordances in design of UIs vs. Gibson view: human as an agent (or robot)
  • Inductive biases in genes
  • Animals only care about a couple of orders of magnitude of space and time (unless they plan and encode complex memories!)
  • There's a natural progression during development: first come border cells, then place cells, and finally grid cells
  • Catching a ball via a heuristic (or doing a high five elbow-to-elbow)

Papers

Chapters 3 and 4

Threads we discussed

  • How animals innately show behavioral responses to some visual features such as a fear of sharp drops. Pointing to "the cliff test," as described in Liz Spelke's 2019 CCN keynote

Papers

Chapter 5

Recording

The ambient optic array

Given a point of observation at some position, the arrangement (heterogeneous structure) of surfaces as conveyed by ambient light in the medium completely surrounding that point.Observer can be inside ambient optic array, but isn’t strictly necessary. When a position is occupied, it now contains information about the body of the observer. This changes the space! Differentiates this framework from perspective in geometry.

The point of observation in ecological optics is never stationary except as a limiting case.

Optic flow

Invariants and occlusion

Although a rectangular surface (e.g., a table) may appear like a trapezoid from a given perspective, you perceive the invariant rectangular shape given different points of observation.

Flowing perspective structure and invariances in structure are coupled.

We perceive whole objects even when they’re occluded by other objects; occluded objects are only temporarily out of sight, so locomotion can resolve uncertainty associated with occlusion (reversibility of occlusion).

“The moving observer and the moving sun are conditions under which terrestrial vision has evolved for millions of years.”

Occluded face culling on the PS1: https://youtu.be/x8TO-nrUtSI?t=286

Tenenbaum paper on physical simulation as an engine for understanding: https://www.pnas.org/content/110/45/18327.short

Chapter 6

Events are not time…

Three kinds of (TERRESTRIAL) events

  1. change in layout of surfaces
  2. change in color and texture of surfaces
  3. change in the existence of surfaces

Events are nested.

Change (over time) as a general concept,

What is common between what happens in the real world and what happens in the optic array?

  • sequences of events (?)
  • temporal synchrony

flash lag experiments

What would Gibson think of Edelman’s concept of representation?

  • internal representations of the world are linked to external events in the world

but Gibson seems to argue that there is no 1-to-1 mapping between state of the world and internal states

however the final outcome (action in the world) might still be valid

Russ Poldrack: do neuroscience and machine learning concepts of representation satisfy philosophical criteria for “representations”?

Gibsonian sense of representation: they just need to encode information about the world even if they do not match “THE REAL WORLD” (whatever that is)

Events define time. There is no arrow of time. Nice link with recent cog neuro experiments on boundaries of events.

This chapter feels human-centered. It’s unclear if all animals, for example, would care about all the changes that are described in chapter 6.

Value of classification: “things that happen on a Tuesday” (does the ability to classify something mean that the class is real (?))

What about water? changes in water, are they events?

Events and time are defined by the sensory systems the animal possesses

Links

Chapter 7:

Field of vision: different for different types of animals; fixed by eyes

Egoception vs exteroception: two sides of the same coin

  • Every sense can be a proprioceptive sense
  • Senses aren’t just specific “nerve energies”; multiple signals need to be combined to extract useful information

Distance isn’t “a line endwise from the eye”:

  • depends on size, disparity, motion perspectives
  • Nose as “baseline”: is it true?
  • Would people overestimate distances on rolling terrain? (Shades of embodied cognition!)

Locomotion produces expansive (“centrifugal) and contractive (“centripedal”) optic flow, except(?) for some invariant structures like the earth/sky

  • Gibson also coined the term "optic flow", which is why Figure 7.4 is in EVERY single lecture about it.
  • How much optic flow happens in real life--plenty of expansion/contraction from locomotion, less translation? (Couldn't find much on this)

Hands provide a "calibration signal" for size:

  • Have a fixed minimum size size (arm at full extension)
  • Developmentally relevant? What happens if you can't see/control limb early in life?
  • Size estimates for objects you can touch, but not see vs see, but not touch.

“Visual kinesthesis” works just like the motor version

  • If you know you’ve moving, you know the world isn’t (or vice versa), even if sensory input is the same.

Does the "ambient optic array" need to account for occulomotor phenomena?

  • Gibson's view of "field of view" changes adds structure at the leading edge, removes it at the trailing edge.
  • In reality, eye movements (and maybe head?) do a lot more: saccadic suppression, release from adaptation, etc.

Rubber hand illusion and the sense of self

Heuristics vs Planning Rule:

Where is Gibson in neuroscience?

  • No 2.5D sketch in neural networks
  • Sam’s Direct-fit paper
  • Gibson as ‘Muse for psychophysicists and robot designers’ vs. Marr’s focus on internals
  • Optic flow is funny since it was coined by Gibson but extensively studied from a Marr-ish perspective.