oci/zeronet
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'master' of github.com:HelloZeroNet/ZeroNet

Browse source
This commit is contained in:
sirMackk 2015-02-22 11:40:44 -05:00
commit de31a518dd
15 changed files with 274 additions and 85 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

112
README.md
View file

@ -1,22 +1,34 @@
# ZeroNet
Decentralized websites using Bitcoin crypto and BitTorrent network
Decentralized websites using Bitcoin crypto and the BitTorrent network
## Why?
- We believe in open, free and uncensored network and communication.
- No single point of failure: Site goes on until at least 1 peer serving it.
- No hosting costs: Site served by visitors.
- Impossible to shut down: It's nowhere because it's everywhere.
- Fast and works offline: You can access the site even if your internet is gone.
* We believe in open, free, and uncensored network and communication.
* No single point of failure: Site remains online so long as at least 1 peer
serving it.
* No hosting costs: Sites are served by visitors.
* Impossible to shut down: It's nowhere because it's everywhere.
* Fast and works offline: You can access the site even if your internet is
unavailable.
## How does it work?
- After starting `zeronet.py` you will be able to visit zeronet sites using http://127.0.0.1:43110/{zeronet_address} (eg. http://127.0.0.1:43110/1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr).
- When you visit a new zeronet site, it's trying to find peers using BitTorrent network and download the site files (html, css, js...) from them.
- Each visited sites become also served by You.
- Every site containing a `site.json` which holds all other files sha512 hash and a sign generated using site's private key.
- If the site owner (who has the private key for the site address) modifies the site, then he/she signs the new `content.json` and publish it to the peers. After the peers have verified the `content.json` integrity (using the sign), they download the modified files and publish the new content to other peers.
* After starting `zeronet.py` you will be able to visit zeronet sites using
`http://127.0.0.1:43110/{zeronet_address}` (eg.
`http://127.0.0.1:43110/1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr`).
* When you visit a new zeronet site, it tries to find peers using the BitTorrent
network so it can download the site files (html, css, js...) from them.
* Each visited site becomes also served by you.
* Every site contains a `site.json` which holds all other files in a sha512 hash
and a signature generated using site's private key.
* If the site owner (who has the private key for the site address) modifies the
site, then he/she signs the new `content.json` and publishes it to the peers.
After the peers have verified the `content.json` integrity (using the
signature), they download the modified files and publish the new content to
other peers.
## Screenshot
@ -25,39 +37,45 @@ Decentralized websites using Bitcoin crypto and BitTorrent network
## How to join?
Windows:
- [Install Python 2.7](https://www.python.org/ftp/python/2.7.9/python-2.7.9.msi)
- [Install Python ZeroMQ](http://zeronet.io/files/windows/pyzmq-14.4.1.win32-py2.7.exe)
- [Install Python Greenlet](http://zeronet.io/files/windows/greenlet-0.4.5.win32-py2.7.exe)
- [Install Python Gevent](http://zeronet.io/files/windows/gevent-1.0.1.win32-py2.7.exe)
- [Install Python MsgPack](http://zeronet.io/files/windows/msgpack-python-0.4.2.win32-py2.7.exe)
- start `start.py`
Linux (Debian):
- `apt-get install python-pip`
- `pip install pyzmq` (if it drops a compile error then run `apt-get install python-dev` and try again)
- `pip install gevent`
- `pip install msgpack-python`
- start using `python zeronet.py`
### Windows
Linux (Without root access):
- `wget https://bootstrap.pypa.io/get-pip.py`
- `python get-pip.py --user pyzmq gevent msgpack-python`
- start using `python zeronet.py`
* [Install Python 2.7](https://www.python.org/ftp/python/2.7.9/python-2.7.9.msi)
* [Install Python ZeroMQ](http://zeronet.io/files/windows/pyzmq-14.4.1.win32-py2.7.exe)
* [Install Python Greenlet](http://zeronet.io/files/windows/greenlet-0.4.5.win32-py2.7.exe)
* [Install Python Gevent](http://zeronet.io/files/windows/gevent-1.0.1.win32-py2.7.exe)
* [Install Python MsgPack](http://zeronet.io/files/windows/msgpack-python-0.4.2.win32-py2.7.exe)
* Start `start.py`
### Linux
#### Debian
* `apt-get install python-dev python-pip`
* `pip install pyzmq gevent msgpack-python`
* Start with `python zeronet.py`
#### Without root access
* `wget https://bootstrap.pypa.io/get-pip.py`
* `python get-pip.py --user pyzmq gevent msgpack-python`
* Start with `python zeronet.py`
## Current limitations
- No torrent-like, file splitting big file support
- Just as anonymous as the bittorrent
- File transactions not compressed or encrypted yet
- No private sites
- You must have an open port to publish new changes
- Timeout problems on slow connections
* No torrent-like, file splitting for big file support
* No more anonymous than Bittorrent
* File transactions are not compressed or encrypted yet
* No private sites
* You must have an open port to publish new changes
* Timeout problems on slow connections
## How can I create a ZeroNet site?
Shut down zeronet if you are running it already
```
```bash
$ zeronet.py siteCreate
...
- Site private key: 23DKQpzxhbVBrAtvLEc2uvk7DZweh4qL3fn3jpM3LgHDczMK2TtYUq
@ -67,27 +85,35 @@ $ zeronet.py siteCreate
$ zeronet.py
...
```
Congratulations, you are done! Now anyone can access your site using http://localhost:43110/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
Congratulations, you're finished! Now anyone can access your site using
`http://localhost:43110/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2`
Next steps: [ZeroNet Developer Documentation](https://github.com/HelloZeroNet/ZeroNet/wiki/ZeroNet-Developer-Documentation)
## How can I modify a ZeroNet site?
- Modify files located in data/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2 directory. After you done:
```
* Modify files located in data/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2 directory.
After you're finished:
```bash
$ zeronet.py siteSign 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
- Signing site: 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2...
Private key (input hidden):
```
- Enter your private key you got when created the site, then:
```
* Enter the private key you got when created the site, then:
```bash
$ zeronet.py sitePublish 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
...
Site:13DNDk..bhC2 Publishing to 3/10 peers...
Site:13DNDk..bhC2 Successfuly published to 3 peers
- Serving files....
```
- That's it! You successfuly signed and published your modifications.
* That's it! You've successfully signed and published your modifications.
## If you want to help keep this project alive
@ -97,5 +123,5 @@ Bitcoin: 1QDhxQ6PraUZa21ET5fYUCPgdrwBomnFgX
#### Thank you!
- More info, help, changelog, zeronet sites: http://www.reddit.com/r/zeronet/
- Come, chat with us: [#zeronet @ FreeNode](https://kiwiirc.com/client/irc.freenode.net/zeronet)
* More info, help, changelog, zeronet sites: http://www.reddit.com/r/zeronet/
* Come, chat with us: [#zeronet @ FreeNode](https://kiwiirc.com/client/irc.freenode.net/zeronet)

4
src/Config.py
View file

@ -3,7 +3,7 @@ import ConfigParser
class Config(object):
def __init__(self):
self.version = "0.2.1"
self.version = "0.2.3"
self.parser = self.createArguments()
argv = sys.argv[:] # Copy command line arguments
argv = self.parseConfig(argv) # Add arguments from config file
@ -19,7 +19,7 @@ class Config(object):
def createArguments(self):
# Platform specific
if sys.platform.startswith("win"):
upnpc = "tools\\upnpc\\upnpc-static.exe"
upnpc = "tools\\upnpc\\upnpc-shared.exe"
coffeescript = "type %s | tools\\coffee\\coffee.cmd"
else:
upnpc = None

20
src/File/FileServer.py
View file

@ -18,6 +18,7 @@ class FileServer:
else:
self.port_opened = None # Is file server opened on router
self.sites = SiteManager.list()
self.running = True
# Handle request to fileserver
@ -168,17 +169,26 @@ class FileServer:
if check_sites: # Open port, Update sites, Check files integrity
gevent.spawn(self.checkSites)
gevent.spawn(self.announceSites)
gevent.spawn(self.wakeupWatcher)
thread_announce_sites = gevent.spawn(self.announceSites)
thread_wakeup_watcher = gevent.spawn(self.wakeupWatcher)
while True:
while self.running:
try:
ret = {}
req = msgpack.unpackb(socket.recv())
self.handleRequest(req)
except Exception, err:
self.log.error(err)
self.socket.send(msgpack.packb({"error": "%s" % Debug.formatException(err)}, use_bin_type=True))
if self.running: self.socket.send(msgpack.packb({"error": "%s" % Debug.formatException(err)}, use_bin_type=True))
if config.debug: # Raise exception
import sys
sys.excepthook(*sys.exc_info())
sys.modules["src.main"].DebugHook.handleError()
thread_wakeup_watcher.kill(exception=Debug.Notify("Stopping FileServer"))
thread_announce_sites.kill(exception=Debug.Notify("Stopping FileServer"))
self.log.debug("Stopped.")
def stop(self):
self.running = False
self.socket.close()

3
src/Site/Site.py
View file

@ -194,9 +194,6 @@ class Site:
self.log.info("[OK] %s: %s" % (peer.key, result["ok"]))
else:
self.log.info("[ERROR] %s: %s" % (peer.key, result))
# Update content.json on peers

36
src/Ui/UiServer.py
View file

@ -11,20 +11,25 @@ from Debug import Debug
# Skip websocket handler if not necessary
class UiWSGIHandler(WSGIHandler):
def __init__(self, *args, **kwargs):
self.server = args[2]
super(UiWSGIHandler, self).__init__(*args, **kwargs)
self.ws_handler = WebSocketHandler(*args, **kwargs)
self.args = args
self.kwargs = kwargs
def run_application(self):
self.server.sockets[self.client_address] = self.socket
if "HTTP_UPGRADE" in self.environ: # Websocket request
self.ws_handler.__dict__ = self.__dict__ # Match class variables
self.ws_handler.run_application()
ws_handler = WebSocketHandler(*self.args, **self.kwargs)
ws_handler.__dict__ = self.__dict__ # Match class variables
ws_handler.run_application()
else: # Standard HTTP request
#print self.application.__class__.__name__
try:
return super(UiWSGIHandler, self).run_application()
except Exception, err:
logging.debug("UiWSGIHandler error: %s" % err)
del self.server.sockets[self.client_address]
class UiServer:
@ -89,4 +94,27 @@ class UiServer:
browser = webbrowser.get(config.open_browser)
browser.open("http://%s:%s" % (config.ui_ip, config.ui_port), new=2)
WSGIServer((self.ip, self.port), handler, handler_class=UiWSGIHandler, log=self.log).serve_forever()
self.server = WSGIServer((self.ip, self.port), handler, handler_class=UiWSGIHandler, log=self.log)
self.server.sockets = {}
self.server.serve_forever()
self.log.debug("Stopped.")
def stop(self):
# Close WS sockets
for client in self.server.clients.values():
client.ws.close()
# Close http sockets
sock_closed = 0
for sock in self.server.sockets.values():
try:
sock._sock.close()
sock.close()
sock_closed += 1
except Exception, err:
pass
self.log.debug("Socket closed: %s" % sock_closed)
self.server.socket.close()
self.server.stop()
time.sleep(1)

11
src/Ui/UiWebsocket.py
View file

@ -121,6 +121,8 @@ class UiWebsocket:
func = self.actionSiteSetLimit
elif cmd == "channelJoinAllsite" and "ADMIN" in permissions:
func = self.actionChannelJoinAllsite
elif cmd == "serverUpdate" and "ADMIN" in permissions:
func = self.actionServerUpdate
# Unknown command
else:
self.response(req["id"], "Unknown command: %s" % cmd)
@ -361,3 +363,12 @@ class UiWebsocket:
self.site.saveSettings()
self.response(to, "Site size limit changed to %sMB" % size_limit)
self.site.download()
def actionServerUpdate(self, to):
import sys
self.cmd("updating")
sys.modules["src.main"].update_after_shutdown = True
sys.modules["src.main"].file_server.stop()
sys.modules["src.main"].ui_server.stop()

19
src/Ui/media/Wrapper.coffee
View file

@ -46,6 +46,9 @@ class Wrapper
@sendInner message # Pass to inner frame
if message.params.address == window.address # Current page
@setSiteInfo message.params
else if cmd == "updating" # Close connection
@ws.ws.close()
@ws.onCloseWebsocket(null, 4000)
else
@sendInner message # Pass message to inner frame
@ -69,6 +72,10 @@ class Wrapper
@actionWrapperPrompt(message)
else if cmd == "wrapperSetViewport" # Set the viewport
@actionSetViewport(message)
else if cmd == "wrapperGetLocalStorage"
@actionGetLocalStorage(message)
else if cmd == "wrapperSetLocalStorage"
@actionSetLocalStorage(message)
else # Send to websocket
if message.id < 1000000
@ws.send(message) # Pass message to websocket
@ -126,6 +133,16 @@ class Wrapper
$('<meta name="viewport" id="viewport">').attr("content", @toHtmlSafe message.params).appendTo("head")
actionGetLocalStorage: (message) ->
data = localStorage.getItem "site.#{window.address}"
if data then data = JSON.parse(data)
@sendInner {"cmd": "response", "to": message.id, "result": data}
actionSetLocalStorage: (message) ->
back = localStorage.setItem "site.#{window.address}", JSON.stringify(message.params)
# EOF actions
@ -152,7 +169,7 @@ class Wrapper
@wrapperWsInited = false
setTimeout (=> # Wait a bit, maybe its page closing
@sendInner {"cmd": "wrapperClosedWebsocket"} # Send to inner frame
if e.code == 1000 # Server error please reload page
if e and e.code == 1000 and e.wasClean == false # Server error please reload page
@ws_error = @notifications.add("connection", "error", "UiServer Websocket error, please reload the page.")
else if not @ws_error
@ws_error = @notifications.add("connection", "error", "Connection with <b>UiServer Websocket</b> was lost. Reconnecting...")

38
src/Ui/media/all.js
View file

@ -120,17 +120,20 @@
}
};
ZeroWebsocket.prototype.onCloseWebsocket = function(e) {
ZeroWebsocket.prototype.onCloseWebsocket = function(e, reconnect) {
if (reconnect == null) {
reconnect = 10000;
}
this.log("Closed", e);
if (e.code === 1000) {
this.log("Server error, please reload the page");
if (e && e.code === 1000 && e.wasClean === false) {
this.log("Server error, please reload the page", e.wasClean);
} else {
setTimeout(((function(_this) {
return function() {
_this.log("Reconnecting...");
return _this.connect();
};
})(this)), 10000);
})(this)), reconnect);
}
if (this.onClose != null) {
return this.onClose(e);
@ -780,6 +783,9 @@ jQuery.extend( jQuery.easing,
if (message.params.address === window.address) {
return this.setSiteInfo(message.params);
}
} else if (cmd === "updating") {
this.ws.ws.close();
return this.ws.onCloseWebsocket(null, 4000);
} else {
return this.sendInner(message);
}
@ -807,6 +813,10 @@ jQuery.extend( jQuery.easing,
return this.actionWrapperPrompt(message);
} else if (cmd === "wrapperSetViewport") {
return this.actionSetViewport(message);
} else if (cmd === "wrapperGetLocalStorage") {
return this.actionGetLocalStorage(message);
} else if (cmd === "wrapperSetLocalStorage") {
return this.actionSetLocalStorage(message);
} else {
if (message.id < 1000000) {
return this.ws.send(message);
@ -891,6 +901,24 @@ jQuery.extend( jQuery.easing,
}
};
Wrapper.prototype.actionGetLocalStorage = function(message) {
var data;
data = localStorage.getItem("site." + window.address);
if (data) {
data = JSON.parse(data);
}
return this.sendInner({
"cmd": "response",
"to": message.id,
"result": data
});
};
Wrapper.prototype.actionSetLocalStorage = function(message) {
var back;
return back = localStorage.setItem("site." + window.address, JSON.stringify(message.params));
};
Wrapper.prototype.onOpenWebsocket = function(e) {
this.ws.cmd("channelJoin", {
"channel": "siteChanged"
@ -925,7 +953,7 @@ jQuery.extend( jQuery.easing,
_this.sendInner({
"cmd": "wrapperClosedWebsocket"
});
if (e.code === 1000) {
if (e && e.code === 1000 && e.wasClean === false) {
return _this.ws_error = _this.notifications.add("connection", "error", "UiServer Websocket error, please reload the page.");
} else if (!_this.ws_error) {
return _this.ws_error = _this.notifications.add("connection", "error", "Connection with <b>UiServer Websocket</b> was lost. Reconnecting...");

8
src/Ui/media/lib/ZeroWebsocket.coffee
View file

@ -66,15 +66,15 @@ class ZeroWebsocket
if @onError? then @onError(e)
onCloseWebsocket: (e) =>
onCloseWebsocket: (e, reconnect=10000) =>
@log "Closed", e
if e.code == 1000
@log "Server error, please reload the page"
if e and e.code == 1000 and e.wasClean == false
@log "Server error, please reload the page", e.wasClean
else # Connection error
setTimeout (=>
@log "Reconnecting..."
@connect()
), 10000
), reconnect
if @onClose? then @onClose(e)

16
src/User/User.py
View file

@ -25,9 +25,9 @@ class User:
self.log.debug("Saved")
# Get BIP32 address from site address
# Return: BIP32 auth address
def getAuthAddress(self, address):
# Get user site data
# Return: {"auth_address": "xxx", "auth_privatekey": "xxx"}
def getSiteData(self, address):
if not address in self.sites: # Genreate new BIP32 child key based on site address
s = time.time()
address_id = int(address.encode("hex"), 16) # Convert site address to int
@ -38,12 +38,17 @@ class User:
}
self.save()
self.log.debug("Added new site: %s in %.3fs" % (address, time.time()-s))
return self.sites[address]
return self.sites[address]["auth_address"]
# Get BIP32 address from site address
# Return: BIP32 auth address
def getAuthAddress(self, address):
return self.getSiteData(address)["auth_address"]
def getAuthPrivatekey(self, address):
return self.sites[address]["auth_privatekey"]
return self.getSiteData(address)["auth_privatekey"]
@ -51,3 +56,4 @@ class User:
def setData(self, data):
for key, val in data.items():
setattr(self, key, val)

2
src/User/UserManager.py
View file

@ -60,6 +60,6 @@ def getCurrent():
def reload():
import imp
global users, User
users.clear() # Remove all items
User = imp.load_source("User", "src/User/User.py").User # Reload source
users.clear() # Remove all items
load()

8
src/Worker/WorkerManager.py
View file

@ -1,5 +1,5 @@
from Worker import Worker
import gevent, time, logging
import gevent, time, logging, random
MAX_WORKERS = 10
@ -87,10 +87,12 @@ class WorkerManager:
def startWorkers(self, peers=None):
if len(self.workers) >= MAX_WORKERS and not peers: return False # Workers number already maxed
if not self.tasks: return False # No task for workers
for key, peer in self.site.peers.iteritems(): # One worker for every peer
peers = self.site.peers.values()
random.shuffle(peers)
for peer in peers: # One worker for every peer
if peers and peer not in peers: continue # If peers definied and peer not valid
worker = self.addWorker(peer)
if worker: self.log.debug("Added worker: %s, workers: %s/%s" % (key, len(self.workers), MAX_WORKERS))
if worker: self.log.debug("Added worker: %s, workers: %s/%s" % (peer.key, len(self.workers), MAX_WORKERS))
# Stop all worker

3
src/main.py
View file

@ -1,4 +1,5 @@
import os, sys
update_after_shutdown = False
sys.path.insert(0, os.path.dirname(__file__)) # Imports relative to main.py
# Create necessary files and dirs
@ -43,7 +44,6 @@ else:
import gevent
import time
logging.debug("Starting... %s" % config)
# Starts here when running zeronet.py
@ -56,6 +56,7 @@ def start():
# Start serving UiServer and PeerServer
def main():
global ui_server, file_server
from File import FileServer
from Ui import UiServer
logging.info("Creating UiServer....")

39
update.py Normal file
View file

@ -0,0 +1,39 @@
from gevent import monkey; monkey.patch_all()
import urllib, zipfile, os, ssl, httplib, socket
import cStringIO as StringIO
def update():
# Gevent https bug workaround (https://github.com/gevent/gevent/issues/477)
reload(socket)
reload(httplib)
reload(ssl)
print "Downloading.",
file = urllib.urlopen("https://github.com/HelloZeroNet/ZeroNet/archive/master.zip")
data = StringIO.StringIO()
while True:
buff = file.read(1024*16)
if not buff: break
data.write(buff)
print ".",
print "Extracting...",
zip = zipfile.ZipFile(data)
for inner_path in zip.namelist():
print ".",
dest_path = inner_path.replace("ZeroNet-master/", "")
if not dest_path: continue
dest_dir = os.path.dirname(dest_path)
if dest_dir and not os.path.isdir(dest_dir):
os.makedirs(dest_dir)
if dest_dir != dest_path.strip("/"):
data = zip.read(inner_path)
open(dest_path, 'wb').write(data)
print "Done."
if __name__ == "__main__":
update()

40
zeronet.py
View file

@ -1,13 +1,37 @@
#!/usr/bin/env python
def main():
try:
from src import main
main.start()
except Exception, err: # Prevent closing
import traceback
traceback.print_exc()
raw_input("-- Error happened, press enter to close --")
try:
from src import main
main.start()
if main.update_after_shutdown: # Updater
import update, sys, os, gc
# Update
update.update()
# Close log files
logger = sys.modules["src.main"].logging.getLogger()
for handler in logger.handlers[:]:
handler.flush()
handler.close()
logger.removeHandler(handler)
except Exception, err: # Prevent closing
import traceback
traceback.print_exc()
raw_input("-- Error happened, press enter to close --")
if main.update_after_shutdown: # Updater
# Restart
gc.collect() # Garbage collect
print "Restarting..."
args = sys.argv[:]
args.insert(0, sys.executable)
if sys.platform == 'win32':
args = ['"%s"' % arg for arg in args]
os.execv(sys.executable, args)
print "Bye."
if __name__ == '__main__':
main()
main()