Skip to content

Commit

Permalink
Merge branch 'main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
ehsan6sha authored Jul 2, 2024
2 parents 25f0838 + fe522fe commit 8484857
Show file tree
Hide file tree
Showing 12 changed files with 81 additions and 39 deletions.
20 changes: 20 additions & 0 deletions .github/styles/pln-ignore.txt
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ CNAME
CNAMEs
codec
Coinlist
Combinator
Composable
composable
config
Expand All @@ -55,10 +56,12 @@ data
data's
Datastore
datastore
dialback
dClimate
deduplicate
Denylist
deserialization
deserialized
Dheeraj
DHT
DHTs
Expand All @@ -67,7 +70,9 @@ discoverability
DNS
DNSAddr
DNSLink
DNSLinks
dnslink
dnslinks
dotgraph
DWeb
dweb
Expand All @@ -78,6 +83,7 @@ ethereum
explainers
Fabien
failover
Fastly
Filebase
Filecoin
filecoin
Expand All @@ -90,6 +96,8 @@ FQDNs
Functionland
gasless
geospatial
Git
Git's
GitHub
Gnutella
goroutine
Expand All @@ -103,11 +111,14 @@ Hareesh
Helia
homebrew
hostname
hostnames
HTML
HTTPS
identafiability
Infura
interop
iotop
iostat
IPFS
IPFS's
ipget
Expand Down Expand Up @@ -152,17 +163,21 @@ metadata's
Metamask
Minty
Minty's
Mojitos
multiaddr
Multiaddrs
Multiaddress
multiaddresses
Multibase
multibase
Multicodec
multicodec
Multicodecs
Multiformat
Multiformats
multiformat
Multihash
Multihashes
multihash
multipage
Nagaraj
Expand Down Expand Up @@ -210,15 +225,19 @@ Rabin
rasterio
README
READMEs
reachability
referenceable
Repo
reprovider
reproviding
retrievability
roadmaps
sandboxed
Satoshi
serverless
sharding
snapshotted
Speedtest
stackparse
stdout
someguy
Expand Down Expand Up @@ -266,3 +285,4 @@ WiFi
WS
WSS
YouTube
Yamux
16 changes: 9 additions & 7 deletions docs/concepts/bitswap.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,11 @@
title: Bitswap
sidebarDepth: 0
description: Learn about Bitswap and how it plays into the overall architecture of IPFS, the InterPlanetary File System.
related:
'Article: Swapping bits and distributing hashes on the decentralized web (Textile)': https://medium.com/textileio/swapping-bits-and-distributing-hashes-on-the-decentralized-web-5da98a3507
'GitHub repo: Go Bitswap implementation': https://github.com/ipfs/go-bitswap
'GitHub repo: JavaScript Bitswap implementation': https://github.com/ipfs/js-ipfs-bitswap
---

# Bitswap

Bitswap is a core module of IPFS for exchanging blocks of data. It directs the requesting and sending of blocks to and from other peers in the network. Bitswap is a _message-based protocol_ where all messages contain [want-lists](#want-lists) or blocks. Bitswap has a [Go implementation](https://github.com/ipfs/go-bitswap) and a [JavaScript implementation](https://github.com/ipfs/js-ipfs-bitswap).
Bitswap is a core module of IPFS for exchanging blocks of data. It directs the requesting and sending of blocks to and from other peers in the network. Bitswap is a _message-based protocol_ where all messages contain [want-lists](#want-lists) or blocks. Bitswap has a [Go implementation](https://github.com/ipfs/boxo/tree/main/bitswap) and a [JavaScript implementation](https://github.com/ipfs/js-ipfs-bitswap).

Bitswap has two main jobs:

Expand Down Expand Up @@ -45,10 +41,16 @@ To find peers that have a file, a node running the Bitswap protocol first sends

Bitswap sends _want-block_ to peers that have the block, and they respond with the block itself. If none of the peers have the root block, Bitswap queries the Distributed Hash Table (DHT) to ask who can provide the root block.

### Technical specification

- [Bitswap Protocol Specifications](https://specs.ipfs.tech/bitswap-protocol/)
- [v1.0.0](https://specs.ipfs.tech/bitswap-protocol/#bitswap-1-0-0)
- [v1.1.0](https://specs.ipfs.tech/bitswap-protocol/#bitswap-1-1-0)
- [v1.2.0](https://specs.ipfs.tech/bitswap-protocol/#bitswap-1-2-0)

### Additional references

- [February 2020: New improvements to IPFS Bitswap](https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-container-distribution/)
- [Technical overview of the Go implementation of Bitswap](https://docs.google.com/presentation/d/1mbFFGIIKNvboHyLn-k26egOSWkt9nXjlNbxpmCEQfqQ/edit#slide=id.p)
- [Article: Swapping bits and distributing hashes on the decentralized web (Textile)](https://medium.com/textileio/swapping-bits-and-distributing-hashes-on-the-decentralized-web-5da98a3507)
- "About Bitswap" Go implementation poster from the IPFS developer summit in Berlin in July 2018:
!['About Bitswap' poster](https://user-images.githubusercontent.com/74178/43230914-f818dab2-901e-11e8-876b-73ba6a084f76.jpg 'Bitswap-Poster_Berlin-July-2018')
- ["About Bitswap" Go implementation poster from the IPFS developer summit in Berlin in July 2018](https://user-images.githubusercontent.com/74178/43230914-f818dab2-901e-11e8-876b-73ba6a084f76.jpg)
4 changes: 2 additions & 2 deletions docs/concepts/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Case-sensitive [Multibase](#multibase), uses modified Base64 with URL and filena

### Bitswap

Bitswap is IPFS's central block exchange protocol. Its purpose is to request blocks from and send blocks to other peers in the network. [More about Bitswap](https://github.com/ipfs/specs/blob/master/BITSWAP.md)
Bitswap is IPFS's central block exchange protocol. Its purpose is to request blocks from and send blocks to other peers in the network. [More about Bitswap](../concepts/bitswap.md)

### BitTorrent

Expand Down Expand Up @@ -166,7 +166,7 @@ Direct Connection Upgrade through Relay (DCUtR) protocol enables [hole punching]

### Delegated routing

Delegated routing is a mechanism by which IPFS implementations can offload content routing, peer routing, and naming (IPNS) to another process/server. The most widely adopted vendor-agnostic spec for delegated routing is the [Delegated Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/).
Delegated routing is a mechanism by which IPFS implementations can offload content routing, peer routing, and naming (IPNS) to another process/server. The most widely adopted vendor-agnostic spec for delegated routing is the [Delegated Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/) with [public utility instance at `delegated-ipfs.dev/routing/v1`](../concepts/public-utilities.md#delegated-routing).

Delegated routing is useful in browsers and other constrained environments where it's infeasible to be a DHT client/server. More broadly, it enables experimentation and innovation in content routing while maintaining interoperability and modularity.

Expand Down
29 changes: 26 additions & 3 deletions docs/concepts/public-utilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ description: Information about hosted public utilities such as public gateways a

The IPFS Foundation (and Protocol Labs in the past) and several other organizations provide hosted public utilities to the community on a best-effort basis. As such, it is not intended to be part of your critical path or production infrastructure.

These include [IPFS Gateways](./ipfs-gateway.md), hosted [Delegated Routing V1 endpoints](./nodes#delegated-routing), and the [Amino DHT](./glossary.md#amino) [Bootstrappers](./nodes.md#bootstrap).
These include [IPFS Gateways](./ipfs-gateway.md), hosted [Delegated Routing V1 endpoints](./glossary.md#delegated-routing), and the [Amino DHT](./glossary.md#amino) [Bootstrappers](./nodes.md#bootstrap).

These utilities make it easier to retrieve data from the IPFS network in resource-constrained environments such as browsers and low-powered devices.

Expand All @@ -34,15 +34,38 @@ From there, an internal system extracts the relevant information from the reques

Your browser may have a local cache of the content in question and might not reflect that something has been blocked on the gateways. To avoid browser caching, attempt to view the content using your browser's incognito or private mode. You can also prevent caching issues by using a command-line tool such as Curl or Wget.

### Supported Features

The above public gateways support the following:

- `ipfs.io` and `dweb.link` support the [full set of IPFS Gateway specifications](https://specs.ipfs.tech/http-gateways/).
- `trustless-gateway.link` supports only the [Trustless Gateway subset of the specification](https://specs.ipfs.tech/http-gateways/trustless-gateway/).

They support HTTP clients reaching them over both ipv4 and ipv6 addresses

The underlying IPFS nodes backing the gateways support the following mutable identifiers under the `/ipns` namespace:
- IPNS Public Keys, e.g. `dweb.link/ipns/k51q....`
- DNSLink for all [ICANN](https://en.wikipedia.org/wiki/ICANN) registered domains as well as `.crypto` and `.eth`, e.g. `dweb.link/ipns/vitalik.eth`

The underlying IPFS nodes backing the gateways support retrieving data from peers that:
- Have either ipv4 or ipv6 addresses
- Are either reachable over the public internet or are accessible via libp2p's relay-v2 protocol and reach out to the gateway nodes via dialback
- Support one of the following libp2p transport configurations:
- QUIC-v1
- TCP or WS or WSS, Yamux, TLS or Noise
- WebTransport
- Support the [Bitswap](./bitswap.md) protocol ([v1.2](https://specs.ipfs.tech/bitswap-protocol/#bitswap-1-2-0), [v1.1](https://specs.ipfs.tech/bitswap-protocol/#bitswap-1-1-0) or [v1.0](https://specs.ipfs.tech/bitswap-protocol/#bitswap-1-0-0))
- Have either advertised their data to the Amino DHT, or have advertised to IPNI such that their data has been indexed by [cid.contact](https://cid.contact)

## Other Public Gateways

Additionally, there's a community-maintained [tool for finding and testing public gateways](https://ipfs.github.io/public-gateway-checker/) such as the one operated by Cloudflare: `https://cf-ipfs.com`.
Additionally, there's a community-maintained [tool for finding and testing public gateways](https://ipfs.github.io/public-gateway-checker/).

## Delegated Routing

While IPFS Gateways are immensely helpful in doing all the heavy lifting of finding providers for CIDs and retrieving them, they can be a choke point for retrieval and a point of centralization.

[_Delegated Routing_](./nodes.md#delegated-routing) endpoints are a key step towards eliminating the emergent centralization of public gateways thereby increasing the health of the network.
[_Delegated Routing_](./glossary.md#delegated-routing) endpoints are a key step towards eliminating the emergent centralization of public gateways thereby increasing the health of the network.

Browsers and low-powered devices can make a single HTTP call to a _Delegated Routing_ endpoint with the CID they are looking to retrieve. The endpoint returns the [multiaddresses](./glossary.md#multiaddr) of the providers for the CID, from which the browser can download directly.

Expand Down
3 changes: 1 addition & 2 deletions docs/how-to/address-ipfs-on-web.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,8 +135,7 @@ Examples:

```plaintext
https://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq.ipfs.dweb.link/wiki/
https://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq.ipfs.cf-ipfs.com/wiki/Vincent_van_Gogh.html
https://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq.ipfs.localhost:8080/wiki/
http://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq.ipfs.localhost:8080/wiki/Vincent_van_Gogh.html
```

#### Native support in Kubo
Expand Down
4 changes: 2 additions & 2 deletions docs/how-to/gateway-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ If you are running an IPFS node that is also configured as an IPFS gateway, each
- Put a CDN like Cloudflare in front of the IPFS gateway.
- Consider enabling the [Accelerated DHT Client](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#accelerated-dht-client).
- Test and monitor your internet connection speed, with a tool like [Speedtest CLI](https://www.speedtest.net/apps/cli).
- Monitor disk I/O and make sure that no other processes are causing disk I/O bottlenecks with a tool like [iotop](https://linux.die.net/man/1/iotop) or [iostat](https://linux.die.net/man/1/iostat).
- Monitor disk I/O and make sure that no other processes are causing disk I/O bottlenecks with a tool like [iotop](https://github.com/Tomas-M/iotop) or [iostat](https://github.com/sysstat/sysstat).



Expand Down Expand Up @@ -89,7 +89,7 @@ A compromised writeable gateway may inject falsified content into the IPFS netwo
1. Alice gives the falsified content CID to Bob.
1. Bob fetches the content with this CID and cryptographically validates the balance of `0.00`.

To partially address this exposure, you may wish to use the public gateway [cf-ipfs.com](https://cf-ipfs.com) as an independent, trusted reference with both same-origin policy and CORS support.
To address this exposure, public gateways should only be limited to [trustless and verifiable response types](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval).

## Assumed filenames when downloading files

Expand Down
6 changes: 2 additions & 4 deletions docs/how-to/nat-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,8 @@ In this step, you will update your Kubo configuration to set `Swarm.AppendAnnoun
"Swarm": [
"/ip4/0.0.0.0/tcp/4001",
"/ip6/::/tcp/4001",
"/ip4/0.0.0.0/udp/4001/quic",
"/ip6/::/udp/4001/quic"
"/ip4/0.0.0.0/udp/4001/quic-v1",
"/ip6/::/udp/4001/quic-v1"
],
"Announce": [],
"AppendAnnounce": [],
Expand All @@ -105,7 +105,6 @@ In this step, you will update your Kubo configuration to set `Swarm.AppendAnnoun
```json
"AppendAnnounce": [
"/ip4/<public-ip>/tcp/<port>",
"/ip4/<public-ip>/udp/<port>/quic",
"/ip4/<public-ip>/udp/<port>/quic-v1",
"/ip4/<public-ip>/udp/<port>/quic-v1/webtransport"
],
Expand All @@ -116,7 +115,6 @@ In this step, you will update your Kubo configuration to set `Swarm.AppendAnnoun
```json
"AppendAnnounce": [
"/ip4/1.2.3.4/tcp/12345",
"/ip4/1.2.3.4/udp/12345/quic",
"/ip4/1.2.3.4/udp/12345/quic-v1",
"/ip4/1.2.3.4/udp/12345/quic-v1/webtransport"
],
Expand Down
6 changes: 3 additions & 3 deletions docs/how-to/peering-with-content-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,9 @@ This list is provided for informational purposes only. The IPFS Project does not

| Peer ID | Addresses |
| ------- | --------- |
|`12D3KooWFFhc8fPYnQXdWBCowxSV21EFYin3rU27p3NVgSMjN41k`|`/ip4/5.161.92.43/tcp/4001`<br/>`/ip4/5.161.92.43/udp/4001/quic`<br/>`/ip6/2a01:4ff:f0:3b1e::1/tcp/4001`<br/>`/ip6/2a01:4ff:f0:3b1e::1/udp/4001/quic`|
|`12D3KooWSW4hoHmDXmY5rW7nCi9XmGTy3foFt72u86jNP53LTNBJ`|`/ip4/5.161.55.227/tcp/4001`<br/>`/ip4/5.161.55.227/udp/4001/quic`<br/>`/ip6/2a01:4ff:f0:1e5a::1/tcp/4001`<br/>`/ip6/2a01:4ff:f0:1e5a::1/udp/4001/quic`|
|`12D3KooWSDj6JM2JmoHwE9AUUwqAFUEg9ndd3pMA8aF2bkYckZfo`|`/ip4/5.161.92.36/tcp/4001`<br/>`/ip4/5.161.92.36/udp/4001/quic`<br/>`/ip6/2a01:4ff:f0:3764::1/tcp/4001`<br/>`/ip6/2a01:4ff:f0:3764::1/udp/4001/quic`|
|`12D3KooWFFhc8fPYnQXdWBCowxSV21EFYin3rU27p3NVgSMjN41k`|`/ip4/5.161.92.43/tcp/4001`<br/>`/ip4/5.161.92.43/udp/4001/quic-v1`<br/>`/ip6/2a01:4ff:f0:3b1e::1/tcp/4001`<br/>`/ip6/2a01:4ff:f0:3b1e::1/udp/4001/quic-v1`|
|`12D3KooWSW4hoHmDXmY5rW7nCi9XmGTy3foFt72u86jNP53LTNBJ`|`/ip4/5.161.55.227/tcp/4001`<br/>`/ip4/5.161.55.227/udp/4001/quic-v1`<br/>`/ip6/2a01:4ff:f0:1e5a::1/tcp/4001`<br/>`/ip6/2a01:4ff:f0:1e5a::1/udp/4001/quic-v1`|
|`12D3KooWSDj6JM2JmoHwE9AUUwqAFUEg9ndd3pMA8aF2bkYckZfo`|`/ip4/5.161.92.36/tcp/4001`<br/>`/ip4/5.161.92.36/udp/4001/quic-v1`<br/>`/ip6/2a01:4ff:f0:3764::1/tcp/4001`<br/>`/ip6/2a01:4ff:f0:3764::1/udp/4001/quic-v1`|

### bit.site

Expand Down
20 changes: 10 additions & 10 deletions docs/how-to/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,15 +50,15 @@ The first thing to do is to double-check that both nodes are, in fact, running a

```json
{
"ID": "QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8",
"PublicKey": "CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDZb6znj3LQZKP1+X81exf+vbnqNCMtHjZ5RKTCm7Fytnfe+AI1fhs9YbZdkgFkM1HLxmIOLQj2bMXPIGxUM+EnewN8tWurx4B3+lR/LWNwNYcCFL+jF2ltc6SE6BC8kMLEZd4zidOLPZ8lIRpd0x3qmsjhGefuRwrKeKlR4tQ3C76ziOms47uLdiVVkl5LyJ5+mn4rXOjNKt/oy2O4m1St7X7/yNt8qQgYsPfe/hCOywxCEIHEkqmil+vn7bu4RpAtsUzCcBDoLUIWuU3i6qfytD05hP8Clo+at+l//ctjMxylf3IQ5qyP+yfvazk+WHcsB0tWueEmiU5P2nfUUIR3AgMBAAE=",
"ID": "12D3KooWRaeAw2oromYUN5rAjYQ6KhqvXiWg8KuxeU9YWv7v3Ewa",
"PublicKey": "CAASp[...]P2nfUUIR3AgMBAAE=",
"Addresses": [
"/ip4/127.0.0.1/tcp/4001/p2p/QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8",
"/ip4/127.0.0.1/udp/4001/quic/p2p/QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8",
"/ip4/192.168.2.131/tcp/4001/p2p/QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8",
"/ip4/192.168.2.131/udp/4001/quic/p2p/QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8"
"/ip4/127.0.0.1/tcp/4001/p2p/12D3KooWRaeAw2oromYUN5rAjYQ6KhqvXiWg8KuxeU9YWv7v3Ewa",
"/ip4/127.0.0.1/udp/4001/quic-v1/p2p/12D3KooWRaeAw2oromYUN5rAjYQ6KhqvXiWg8KuxeU9YWv7v3Ewa",
"/ip4/192.168.2.131/tcp/4001/p2p/12D3KooWRaeAw2oromYUN5rAjYQ6KhqvXiWg8KuxeU9YWv7v3Ewa",
"/ip4/192.168.2.131/udp/4001/quic-v1/p2p/12D3KooWRaeAw2oromYUN5rAjYQ6KhqvXiWg8KuxeU9YWv7v3Ewa"
],
"AgentVersion": "kubo/0.4.11-dev/",
"AgentVersion": "kubo/0.29.0-dev/",
"ProtocolVersion": "ipfs/0.1.0"
}
```
Expand All @@ -75,11 +75,11 @@ In the case where `node b` simply cannot form a connection to `node a`, despite

```shell
/ip4/127.0.0.1/tcp/4001
/ip4/127.0.0.1/udp/4001/quic
/ip4/127.0.0.1/udp/4001/quic-v1
/ip4/192.168.2.133/tcp/4001
/ip4/192.168.2.133/udp/4001/quic
/ip4/192.168.2.133/udp/4001/quic-v1
/ip4/88.157.217.196/tcp/63674
/ip4/88.157.217.196/udp/63674/quic
/ip4/88.157.217.196/udp/63674/quic-v1
```

In this case, we can see a localhost (127.0.0.1) address, a LAN address (the 192.168._._ one), and another address. If this third address matches your external IP, then the network knows a valid external address for your node. At this point, it's safe to assume that your node has a difficult to traverse NAT situation. If this is the case, you can try to enable UPnP or NAT-PMP on the router of `node a` and retry the process. Otherwise, you can try manually connecting `node a` to `node b`.
Expand Down
Loading

0 comments on commit 8484857

Please sign in to comment.