Compare commits

...

51 commits

Author SHA1 Message Date
shortcutme
bf771eda5f
Restrict setting open_browser values in config file 2019-08-28 01:33:32 +02:00
shortcutme
a9b5561c49
Rev3870 2019-08-28 01:32:28 +02:00
shortcutme
a121c23973
Use re.sub to replace template variables 2019-08-28 01:32:16 +02:00
shortcutme
27a67d9753
Allow websocket connection originates from earlier accepted hostnames 2019-08-28 01:32:02 +02:00
shortcutme
67b78ca12d
Rev3868, Add origin validation to websocket connections 2019-08-18 03:20:44 +02:00
ZeroNet
77a5d88ec9
Merge pull request #2082 from kusky33/patch-1
Update it.json
2019-07-07 20:01:43 +02:00
kusky
861e085597
Update it.json 2019-07-07 15:11:53 +02:00
ZeroNet
32e9839f23
Merge pull request #2074 from HughIsaacs2/patch-1
Update Sidebar.css for grabbing cursor
2019-07-06 13:24:27 +02:00
Hugh Isaacs II
028d491294
Update Sidebar.css
Cross browser grabbing cursor.
2019-07-05 14:42:23 -04:00
ZeroNet
bdca28844c
Merge pull request #2031 from filips123/patch-2
Display a sponsor button in repository
2019-07-04 18:56:24 +02:00
ZeroNet
9e7ae55068
Merge pull request #1453 from anoadragon453/anoa/fix_filepaths
Only raise security error on ../
2019-06-30 16:46:55 +02:00
shortcutme
7418400b52
Rev3866, Fix and test serving files with null bytes 2019-06-06 02:31:08 +02:00
shortcutme
c165d21d95
Rev3865, Fix ZipStream seek support 2019-05-31 15:04:03 +02:00
Filip Š
f08bea7f90
Create FUNDING.yml 2019-05-24 18:01:22 +02:00
shortcutme
9b274415e0
Rev3864, Fix newsfeed sql query with many parameters 2019-04-29 16:36:33 +02:00
ZeroNet
8dd3a8495b
Merge pull request #1995 from ValdikSS/translation-fix
Always translate html files to avoid compatibility problems with brackets in url
2019-04-23 02:02:25 +02:00
ValdikSS
fd7f724e2b Always translate html files to avoid compatibility problems with brackets in url
Backport of commit 90fee9788d from py3 branch
2019-04-22 01:16:36 +03:00
shortcutme
3366edc244
Rev3863 2019-04-19 02:38:41 +02:00
shortcutme
129aff2c0c
Retry peers only once 2019-04-19 02:38:02 +02:00
shortcutme
7e78fbc16e
Ignore invalid shared filters 2019-04-19 02:37:27 +02:00
shortcutme
719df4ac88
Rev3862 2019-04-11 02:28:38 +02:00
shortcutme
7a217a3741
Only display error details in debug mode 2019-04-11 02:28:26 +02:00
shortcutme
85fd08774f
Send noscript header for error message pages 2019-04-11 02:28:01 +02:00
shortcutme
c0d81021df
Rev3861, Escape error detail to avoid XSS (reported by krzotr) 2019-04-11 00:48:16 +02:00
ZeroNet
5d81467083
Merge pull request #1956 from krzotr/master
OptionalFileList - Added support for filter not_downloaded
2019-04-09 17:53:23 +02:00
shortcutme
7a59a19df1
Fix double --open_browser (by imachug) 2019-04-07 12:11:30 +02:00
ZeroNet
171b591798
Merge pull request #1957 from krzotr/patch-1
Added support for encrypted connections in Dockerfile - added OpenSSL…
2019-04-04 19:19:45 +02:00
krzotr
27c47bb3bd
Added support for encrypted connections in Dockerfile - added OpenSSL library 2019-04-04 18:39:27 +02:00
krzotr
80f3f9d511 OptionalFileList - get list of not downloaded files
We can use API command optionalFileList with parameter filter=not_downloaded to get list of all not downloaded files
2019-04-03 17:05:02 +02:00
ZeroNet
ba6a75f8d7
Merge pull request #1939 from tangdou1/patch-3
sitePause & siteResume are also important settings
2019-03-27 12:45:00 +01:00
tangdou1
b3f677f806
sitePause & siteResume are also important settings 2019-03-27 11:06:41 +08:00
ZeroNet
c51dfe728f
Merge pull request #1938 from tangdou1/patch-2
a bug?
2019-03-27 03:07:33 +01:00
shortcutme
eb88dbbec8
Rev3860 2019-03-27 03:01:56 +01:00
shortcutme
350adeb52d
Fix resource loading with origin only referer 2019-03-27 03:01:39 +01:00
shortcutme
5ab20317d0
Fix ssl compatibility with older clients, prefer chacha20-poly1305 if possible 2019-03-27 03:00:44 +01:00
shortcutme
cdd0f9cda3
Remove srl file 2019-03-27 02:59:57 +01:00
shortcutme
d504cdf501
Formatting CryptConnection.py 2019-03-27 02:59:41 +01:00
tangdou1
e333b47c27
maybe a bug 2019-03-27 09:44:36 +08:00
ZeroNet
91b2f6a8a7
Merge pull request #1928 from ValdikSS/crypt-obf
Less obvious fake TLS certificate generation
2019-03-27 02:43:23 +01:00
ValdikSS
f66cfc9a5e Less obvious fake TLS certificate generation
This patch adds the following:
 * Pre-defined CA certificate subjects
 * Pre-defined popular website domain names
 * Fake certificate generation for pre-defined popular website domain signed by fake CA with pre-defined subject

It should look less suspicious than "example.com" certificates
2019-03-27 00:29:01 +03:00
ZeroNet
911733955b
Merge pull request #1927 from ValdikSS/update-ciphers
Update ciphersuites
2019-03-25 17:15:43 +01:00
shortcutme
abb566e35f
Merge branch 'master' of https://github.com/HelloZeroNet/ZeroNet 2019-03-23 03:35:21 +01:00
shortcutme
06be430b74
Rev3857 2019-03-23 03:35:13 +01:00
shortcutme
ef892e91da
Add random padding to handshake 2019-03-23 03:34:27 +01:00
shortcutme
c198237938
Remove signal watcher from zeronet.py 2019-03-23 03:33:41 +01:00
shortcutme
74ce0c50ff
Proper shutdown at sigterm 2019-03-23 03:33:27 +01:00
shortcutme
b8b8ce21fa
Pass kwargs to excepthook 2019-03-23 03:33:12 +01:00
shortcutme
5716b7505f
Add reason for shutdown 2019-03-23 03:32:09 +01:00
shortcutme
b4ceb6957c
Fix return value of bigfile upload post request 2019-03-23 03:31:35 +01:00
ValdikSS
d51e9c68f4 Update ciphersuites 2019-03-19 18:24:11 +03:00
user
4d25a02cd1 Only raise security error on ../ 2018-06-10 15:35:34 +01:00
27 changed files with 254 additions and 133 deletions

1
.github/FUNDING.yml vendored Normal file
View file

@ -0,0 +1 @@
custom: https://zeronet.io/docs/help_zeronet/donate/

View file

@ -4,7 +4,7 @@ FROM alpine:3.8
ENV HOME /root ENV HOME /root
#Install ZeroNet #Install ZeroNet
RUN apk --no-cache --no-progress add musl-dev gcc python python-dev py2-pip tor \ RUN apk --no-cache --no-progress add musl-dev gcc python python-dev py2-pip tor openssl \
&& pip install --no-cache-dir gevent msgpack \ && pip install --no-cache-dir gevent msgpack \
&& apk del musl-dev gcc python-dev py2-pip \ && apk del musl-dev gcc python-dev py2-pip \
&& echo "ControlPort 9051" >> /etc/tor/torrc \ && echo "ControlPort 9051" >> /etc/tor/torrc \

View file

@ -4,6 +4,7 @@ import subprocess
import shutil import shutil
import collections import collections
import math import math
import json
import msgpack import msgpack
import gevent import gevent
@ -96,12 +97,12 @@ class UiRequestPlugin(object):
site.content_manager.contents.loadItem(file_info["content_inner_path"]) # reload cache site.content_manager.contents.loadItem(file_info["content_inner_path"]) # reload cache
return { return json.dumps({
"merkle_root": merkle_root, "merkle_root": merkle_root,
"piece_num": len(piecemap_info["sha512_pieces"]), "piece_num": len(piecemap_info["sha512_pieces"]),
"piece_size": piece_size, "piece_size": piece_size,
"inner_path": inner_path "inner_path": inner_path
} })
def readMultipartHeaders(self, wsgi_input): def readMultipartHeaders(self, wsgi_input):
for i in range(100): for i in range(100):
@ -604,6 +605,7 @@ class FileRequestPlugin(object):
if file.read(10) == "\0" * 10: if file.read(10) == "\0" * 10:
# Looks empty, but makes sures we don't have that piece # Looks empty, but makes sures we don't have that piece
file_info = site.content_manager.getFileInfo(inner_path) file_info = site.content_manager.getFileInfo(inner_path)
if "piece_size" in file_info:
piece_i = pos / file_info["piece_size"] piece_i = pos / file_info["piece_size"]
if not site.storage.piecefields[file_info["sha512"]][piece_i]: if not site.storage.piecefields[file_info["sha512"]][piece_i]:
return False return False

View file

@ -491,3 +491,32 @@ class TestBigfile:
site_temp.needFile("%s|%s-%s" % (inner_path, 9 * 1024 * 1024, 10 * 1024 * 1024)) site_temp.needFile("%s|%s-%s" % (inner_path, 9 * 1024 * 1024, 10 * 1024 * 1024))
assert site_temp.storage.getSize(inner_path) == site.storage.getSize(inner_path) assert site_temp.storage.getSize(inner_path) == site.storage.getSize(inner_path)
@pytest.mark.parametrize("size", [1024 * 3, 1024 * 1024 * 3, 1024 * 1024 * 30])
def testNullFileRead(self, file_server, site, site_temp, size):
inner_path = "data/optional.iso"
f = site.storage.open(inner_path, "w")
f.write("\0" * size)
f.close()
assert site.content_manager.sign("content.json", self.privatekey)
# Init source server
site.connection_server = file_server
file_server.sites[site.address] = site
# Init client server
site_temp.connection_server = FileServer(file_server.ip, 1545)
site_temp.connection_server.sites[site_temp.address] = site_temp
site_temp.addPeer(file_server.ip, 1544)
# Download site
site_temp.download(blind_includes=True).join(timeout=5)
if "piecemap" in site.content_manager.getFileInfo(inner_path): # Bigfile
site_temp.needFile(inner_path + "|all")
else:
site_temp.needFile(inner_path)
assert site_temp.storage.getSize(inner_path) == size

View file

@ -145,7 +145,14 @@ class UiWebsocketPlugin(object):
include_site = filter_storage.site_manager.get(include["address"]) include_site = filter_storage.site_manager.get(include["address"])
if not include_site: if not include_site:
continue continue
try:
content = include_site.storage.loadJson(include["inner_path"]) content = include_site.storage.loadJson(include["inner_path"])
include["error"] = None
except Exception as err:
if include_site.settings["own"]:
include_site.log.warning("Error loading filter %s: %s" % (include["inner_path"], err))
content = {}
include["error"] = str(err)
include["mutes"] = content.get("mutes", {}) include["mutes"] = content.get("mutes", {})
include["siteblocks"] = content.get("siteblocks", {}) include["siteblocks"] = content.get("siteblocks", {})
back.append(include) back.append(include)

View file

@ -4,6 +4,7 @@ import re
from Plugin import PluginManager from Plugin import PluginManager
from Db import DbQuery from Db import DbQuery
from Debug import Debug from Debug import Debug
from util import helper
@PluginManager.registerTo("UiWebsocket") @PluginManager.registerTo("UiWebsocket")
@ -66,14 +67,14 @@ class UiWebsocketPlugin(object):
query = " UNION ".join(query_parts) query = " UNION ".join(query_parts)
if ":params" in query: if ":params" in query:
query = query.replace(":params", ",".join(["?"] * len(params))) query_params = map(helper.sqlquote, params)
res = site.storage.query(query + " ORDER BY date_added DESC LIMIT %s" % limit, params * query_raw.count(":params")) query = query.replace(":params", ",".join(query_params))
else:
res = site.storage.query(query + " ORDER BY date_added DESC LIMIT %s" % limit) res = site.storage.query(query + " ORDER BY date_added DESC LIMIT %s" % limit)
except Exception as err: # Log error except Exception as err: # Log error
self.log.error("%s feed query %s error: %s" % (address, name, Debug.formatException(err))) self.log.error("%s feed query %s error: %s" % (address, name, Debug.formatException(err)))
stats.append({"site": site.address, "feed_name": name, "error": str(err), "query": query}) stats.append({"site": site.address, "feed_name": name, "error": str(err)})
continue continue
for row in res: for row in res:

View file

@ -132,8 +132,12 @@ class UiWebsocketPlugin(object):
wheres_raw = [] wheres_raw = []
if "bigfile" in filter: if "bigfile" in filter:
wheres["size >"] = 1024 * 1024 * 10 wheres["size >"] = 1024 * 1024 * 10
if "downloaded" in filter:
if "not_downloaded" in filter:
wheres["is_downloaded"] = 0
elif "downloaded" in filter:
wheres_raw.append("(is_downloaded = 1 OR is_pinned = 1)") wheres_raw.append("(is_downloaded = 1 OR is_pinned = 1)")
if "pinned" in filter: if "pinned" in filter:
wheres["is_pinned"] = 1 wheres["is_pinned"] = 1

View file

@ -2,11 +2,11 @@ import cStringIO as StringIO
import os import os
import zipfile import zipfile
class ZipStream(file): class ZipStream(file):
def __init__(self, dir_path): def __init__(self, dir_path):
self.dir_path = dir_path self.dir_path = dir_path
self.pos = 0 self.pos = 0
self.buff_pos = 0
self.zf = zipfile.ZipFile(self, 'w', zipfile.ZIP_DEFLATED, allowZip64=True) self.zf = zipfile.ZipFile(self, 'w', zipfile.ZIP_DEFLATED, allowZip64=True)
self.buff = StringIO.StringIO() self.buff = StringIO.StringIO()
self.file_list = self.getFileList() self.file_list = self.getFileList()
@ -27,6 +27,8 @@ class ZipStream(file):
self.buff.seek(0) self.buff.seek(0)
back = self.buff.read() back = self.buff.read()
self.buff.truncate(0) self.buff.truncate(0)
self.buff.seek(0)
self.buff_pos += len(back)
return back return back
def write(self, data): def write(self, data):
@ -36,8 +38,22 @@ class ZipStream(file):
def tell(self): def tell(self):
return self.pos return self.pos
def seek(self, pos, type): def seek(self, pos, whence=0):
pass if pos >= self.buff_pos:
self.buff.seek(pos - self.buff_pos, whence)
self.pos = pos
def flush(self): def flush(self):
pass pass
if __name__ == "__main__":
zs = ZipStream(".")
out = open("out.zip", "wb")
while 1:
data = zs.read()
print("Write %s" % len(data))
if not data:
break
out.write(data)
out.close()

View file

@ -3,8 +3,8 @@
} }
.drag-bg { width: 100%; height: 100%; position: fixed; } .drag-bg { width: 100%; height: 100%; position: fixed; }
.fixbutton.dragging { cursor: -webkit-grabbing; } .fixbutton.dragging { cursor: -webkit-grabbing; cusor: grabbing; }
.fixbutton-bg:active { cursor: -webkit-grabbing; } .fixbutton-bg:active { cursor: -webkit-grabbing; cusor: grabbing; }
.body-sidebar, .body-internals { background-color: #666 !important; } .body-sidebar, .body-internals { background-color: #666 !important; }

View file

@ -7,11 +7,18 @@ from Translate import translate
@PluginManager.registerTo("UiRequest") @PluginManager.registerTo("UiRequest")
class UiRequestPlugin(object): class UiRequestPlugin(object):
def actionSiteMedia(self, path, **kwargs): def actionSiteMedia(self, path, **kwargs):
file_name = path.split("/")[-1] file_name = path.split("/")[-1].lower()
if not file_name: # Path ends with / if not file_name: # Path ends with /
file_name = "index.html" file_name = "index.html"
extension = file_name.split(".")[-1] extension = file_name.split(".")[-1]
if translate.lang != "en" and extension in ["js", "html"]: if extension == "html":
should_translate = True
elif extension == "js" and translate.lang != "en":
should_translate = True
else:
should_translate = False
if should_translate:
path_parts = self.parsePath(path) path_parts = self.parsePath(path)
kwargs["header_length"] = False kwargs["header_length"] = False
file_generator = super(UiRequestPlugin, self).actionSiteMedia(path, **kwargs) file_generator = super(UiRequestPlugin, self).actionSiteMedia(path, **kwargs)

View file

@ -96,7 +96,7 @@ class UiRequestPlugin(object):
class UiWebsocketPlugin(object): class UiWebsocketPlugin(object):
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
self.multiuser_denied_cmds = ( self.multiuser_denied_cmds = (
"siteDelete", "configSet", "serverShutdown", "serverUpdate", "siteClone", "sitePause", "siteResume", "siteDelete", "configSet", "serverShutdown", "serverUpdate", "siteClone",
"siteSetOwned", "siteSetAutodownloadoptional", "dbReload", "dbRebuild", "siteSetOwned", "siteSetAutodownloadoptional", "dbReload", "dbRebuild",
"mergerSiteDelete", "siteSetLimit", "siteSetAutodownloadBigfileLimit", "mergerSiteDelete", "siteSetLimit", "siteSetAutodownloadBigfileLimit",
"optionalLimitSet", "optionalHelp", "optionalHelpRemove", "optionalHelpAll", "optionalFilePin", "optionalFileUnpin", "optionalFileDelete", "optionalLimitSet", "optionalHelp", "optionalHelpRemove", "optionalHelpAll", "optionalFilePin", "optionalFileUnpin", "optionalFileDelete",

View file

@ -13,7 +13,7 @@ class Config(object):
def __init__(self, argv): def __init__(self, argv):
self.version = "0.6.5" self.version = "0.6.5"
self.rev = 3853 self.rev = 3870
self.argv = argv self.argv = argv
self.action = None self.action = None
self.pending_changes = {} self.pending_changes = {}

View file

@ -1,5 +1,6 @@
import socket import socket
import time import time
import random
import gevent import gevent
import msgpack import msgpack
@ -172,7 +173,7 @@ class Connection(object):
self.sock.connect(sock_address) self.sock.connect(sock_address)
# Detect protocol # Detect protocol
self.send({"cmd": "handshake", "req_id": 0, "params": self.getHandshakeInfo()}) self.send({"cmd": "handshake", "req_id": 0, "params": self.getHandshakeInfo(), "random": "A" * random.randint(0, 1024)})
event_connected = self.event_connected event_connected = self.event_connected
gevent.spawn(self.messageLoop) gevent.spawn(self.messageLoop)
connect_res = event_connected.get() # Wait for handshake connect_res = event_connected.get() # Wait for handshake

View file

@ -3,6 +3,7 @@ import logging
import os import os
import ssl import ssl
import hashlib import hashlib
import random
from Config import config from Config import config
from util import SslPatch from util import SslPatch
@ -20,6 +21,12 @@ class CryptConnectionManager:
self.crypt_supported = [] # Supported cryptos self.crypt_supported = [] # Supported cryptos
self.cacert_pem = config.data_dir + "/cacert-rsa.pem"
self.cakey_pem = config.data_dir + "/cakey-rsa.pem"
self.cert_pem = config.data_dir + "/cert-rsa.pem"
self.cert_csr = config.data_dir + "/cert-rsa.csr"
self.key_pem = config.data_dir + "/key-rsa.pem"
# Select crypt that supported by both sides # Select crypt that supported by both sides
# Return: Name of the crypto # Return: Name of the crypto
def selectCrypt(self, client_supported): def selectCrypt(self, client_supported):
@ -32,12 +39,13 @@ class CryptConnectionManager:
# Return: wrapped socket # Return: wrapped socket
def wrapSocket(self, sock, crypt, server=False, cert_pin=None): def wrapSocket(self, sock, crypt, server=False, cert_pin=None):
if crypt == "tls-rsa": if crypt == "tls-rsa":
ciphers = "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:AES128-GCM-SHA256:AES128-SHA256:HIGH:" ciphers = "ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:AES128-SHA256:AES256-SHA:"
ciphers += "!aNULL:!eNULL:!EXPORT:!DSS:!DES:!RC4:!3DES:!MD5:!PSK" ciphers += "!aNULL:!eNULL:!EXPORT:!DSS:!DES:!RC4:!3DES:!MD5:!PSK"
if server: if server:
sock_wrapped = ssl.wrap_socket( sock_wrapped = ssl.wrap_socket(
sock, server_side=server, keyfile='%s/key-rsa.pem' % config.data_dir, sock, server_side=server, keyfile=self.key_pem,
certfile='%s/cert-rsa.pem' % config.data_dir, ciphers=ciphers) certfile=self.cert_pem, ciphers=ciphers
)
else: else:
sock_wrapped = ssl.wrap_socket(sock, ciphers=ciphers) sock_wrapped = ssl.wrap_socket(sock, ciphers=ciphers)
if cert_pin: if cert_pin:
@ -50,7 +58,7 @@ class CryptConnectionManager:
def removeCerts(self): def removeCerts(self):
if config.keep_ssl_cert: if config.keep_ssl_cert:
return False return False
for file_name in ["cert-rsa.pem", "key-rsa.pem"]: for file_name in ["cert-rsa.pem", "key-rsa.pem", "cacert-rsa.pem", "cakey-rsa.pem", "cacert-rsa.srl", "cert-rsa.csr"]:
file_path = "%s/%s" % (config.data_dir, file_name) file_path = "%s/%s" % (config.data_dir, file_name)
if os.path.isfile(file_path): if os.path.isfile(file_path):
os.unlink(file_path) os.unlink(file_path)
@ -66,15 +74,33 @@ class CryptConnectionManager:
# Try to create RSA server cert + sign for connection encryption # Try to create RSA server cert + sign for connection encryption
# Return: True on success # Return: True on success
def createSslRsaCert(self): def createSslRsaCert(self):
if os.path.isfile("%s/cert-rsa.pem" % config.data_dir) and os.path.isfile("%s/key-rsa.pem" % config.data_dir): casubjects = [
"/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon",
"/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3",
"/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA",
"/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA"
]
fakedomains = [
"yahoo.com", "amazon.com", "live.com", "microsoft.com", "mail.ru", "csdn.net", "bing.com",
"amazon.co.jp", "office.com", "imdb.com", "msn.com", "samsung.com", "huawei.com", "ztedevices.com",
"godaddy.com", "w3.org", "gravatar.com", "creativecommons.org", "hatena.ne.jp",
"adobe.com", "opera.com", "apache.org", "rambler.ru", "one.com", "nationalgeographic.com",
"networksolutions.com", "php.net", "python.org", "phoca.cz", "debian.org", "ubuntu.com",
"nazwa.pl", "symantec.com"
]
self.openssl_env['CN'] = random.choice(fakedomains)
if os.path.isfile(self.cert_pem) and os.path.isfile(self.key_pem):
return True # Files already exits return True # Files already exits
import subprocess import subprocess
cmd = "%s req -x509 -newkey rsa:2048 -sha256 -batch -keyout %s -out %s -nodes -config %s" % helper.shellquote( # Generate CAcert and CAkey
cmd = "%s req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj %s -keyout %s -out %s -batch -config %s" % helper.shellquote(
self.openssl_bin, self.openssl_bin,
config.data_dir+"/key-rsa.pem", random.choice(casubjects),
config.data_dir+"/cert-rsa.pem", self.cakey_pem,
self.openssl_env["OPENSSL_CONF"] self.cacert_pem,
self.openssl_env["OPENSSL_CONF"],
) )
proc = subprocess.Popen( proc = subprocess.Popen(
cmd.encode(sys.getfilesystemencoding()), cmd.encode(sys.getfilesystemencoding()),
@ -82,47 +108,50 @@ class CryptConnectionManager:
) )
back = proc.stdout.read().strip() back = proc.stdout.read().strip()
proc.wait() proc.wait()
logging.debug("Generating RSA cert and key PEM files...%s" % back) logging.debug("Generating RSA CAcert and CAkey PEM files...%s" % back)
if os.path.isfile("%s/cert-rsa.pem" % config.data_dir) and os.path.isfile("%s/key-rsa.pem" % config.data_dir): if not (os.path.isfile(self.cacert_pem) and os.path.isfile(self.cakey_pem)):
logging.error("RSA ECC SSL CAcert generation failed, CAcert or CAkey files not exist.")
return False
# Generate certificate key and signing request
cmd = "%s req -new -newkey rsa:2048 -keyout %s -out %s -subj %s -sha256 -nodes -batch -config %s" % helper.shellquote(
self.openssl_bin,
self.key_pem,
self.cert_csr,
"/CN=" + self.openssl_env['CN'],
self.openssl_env["OPENSSL_CONF"],
)
proc = subprocess.Popen(
cmd.encode(sys.getfilesystemencoding()),
shell=True, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, env=self.openssl_env
)
back = proc.stdout.read().strip()
proc.wait()
logging.debug("Generating certificate key and signing request...%s" % back)
# Sign request and generate certificate
cmd = "%s x509 -req -in %s -CA %s -CAkey %s -CAcreateserial -out %s -days 730 -sha256 -extensions x509_ext -extfile %s" % helper.shellquote(
self.openssl_bin,
self.cert_csr,
self.cacert_pem,
self.cakey_pem,
self.cert_pem,
self.openssl_env["OPENSSL_CONF"],
)
proc = subprocess.Popen(
cmd.encode(sys.getfilesystemencoding()),
shell=True, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, env=self.openssl_env
)
back = proc.stdout.read().strip()
proc.wait()
logging.debug("Generating RSA cert...%s" % back)
if os.path.isfile(self.cert_pem) and os.path.isfile(self.key_pem):
return True return True
else: else:
logging.error("RSA ECC SSL cert generation failed, cert or key files not exist.") logging.error("RSA ECC SSL cert generation failed, cert or key files not exist.")
return False return False
# Not used yet: Missing on some platform
"""def createSslEccCert(self):
return False
import subprocess
# Create ECC privatekey
proc = subprocess.Popen(
"%s ecparam -name prime256v1 -genkey -out %s/key-ecc.pem" % (self.openssl_bin, config.data_dir),
shell=True, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, env=self.openssl_env
)
back = proc.stdout.read().strip()
proc.wait()
self.log.debug("Generating ECC privatekey PEM file...%s" % back)
# Create ECC cert
proc = subprocess.Popen(
"%s req -new -key %s -x509 -nodes -out %s -config %s" % helper.shellquote(
self.openssl_bin,
config.data_dir+"/key-ecc.pem",
config.data_dir+"/cert-ecc.pem",
self.openssl_env["OPENSSL_CONF"]
),
shell=True, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, env=self.openssl_env
)
back = proc.stdout.read().strip()
proc.wait()
self.log.debug("Generating ECC cert PEM file...%s" % back)
if os.path.isfile("%s/cert-ecc.pem" % config.data_dir) and os.path.isfile("%s/key-ecc.pem" % config.data_dir):
return True
else:
self.logging.error("ECC SSL cert generation failed, cert or key files not exits.")
return False
"""
manager = CryptConnectionManager() manager = CryptConnectionManager()

View file

@ -1,5 +1,7 @@
import time import time
import re import re
from util import helper
# Special sqlite cursor # Special sqlite cursor
@ -12,12 +14,6 @@ class DbCursor:
self.cursor = conn.cursor() self.cursor = conn.cursor()
self.logging = False self.logging = False
def quoteValue(self, value):
if type(value) is int:
return str(value)
else:
return "'%s'" % value.replace("'", "''")
def execute(self, query, params=None): def execute(self, query, params=None):
self.db.last_query_time = time.time() self.db.last_query_time = time.time()
if isinstance(params, dict) and "?" in query: # Make easier select and insert by allowing dict params if isinstance(params, dict) and "?" in query: # Make easier select and insert by allowing dict params
@ -35,7 +31,7 @@ class DbCursor:
operator = "IN" operator = "IN"
if len(value) > 100: if len(value) > 100:
# Embed values in query to avoid "too many SQL variables" error # Embed values in query to avoid "too many SQL variables" error
query_values = ",".join(map(self.quoteValue, value)) query_values = ",".join(map(helper.sqlquote, value))
else: else:
query_values = ",".join(["?"] * len(value)) query_values = ",".join(["?"] * len(value))
values += value values += value

View file

@ -1,5 +1,6 @@
import sys import sys
import logging import logging
import signal
import gevent import gevent
import gevent.hub import gevent.hub
@ -8,22 +9,22 @@ from Config import config
last_error = None last_error = None
def shutdown(): def shutdown(reason="Unknown"):
print "Shutting down..." logging.info("Shutting down (reason: %s)..." % reason)
if "file_server" in dir(sys.modules["main"]) and sys.modules["main"].file_server.running: if "file_server" in dir(sys.modules["main"]) and sys.modules["main"].file_server.running:
try: try:
if "file_server" in dir(sys.modules["main"]): if "file_server" in dir(sys.modules["main"]):
gevent.spawn(sys.modules["main"].file_server.stop) gevent.spawn(sys.modules["main"].file_server.stop)
if "ui_server" in dir(sys.modules["main"]): if "ui_server" in dir(sys.modules["main"]):
gevent.spawn(sys.modules["main"].ui_server.stop) gevent.spawn(sys.modules["main"].ui_server.stop)
except Exception, err: except Exception as err:
print "Proper shutdown error: %s" % err print "Proper shutdown error: %s" % err
sys.exit(0) sys.exit(0)
else: else:
sys.exit(0) sys.exit(0)
# Store last error, ignore notify, allow manual error logging # Store last error, ignore notify, allow manual error logging
def handleError(*args): def handleError(*args, **kwargs):
global last_error global last_error
if not args: # Manual called if not args: # Manual called
args = sys.exc_info() args = sys.exc_info()
@ -32,22 +33,23 @@ def handleError(*args):
silent = False silent = False
if args[0].__name__ != "Notify": if args[0].__name__ != "Notify":
last_error = args last_error = args
if args[0].__name__ == "KeyboardInterrupt": if args[0].__name__ == "KeyboardInterrupt":
shutdown() shutdown("Keyboard interrupt")
return elif not silent and args[0].__name__ != "Notify":
if not silent and args[0].__name__ != "Notify":
logging.exception("Unhandled exception") logging.exception("Unhandled exception")
if "greenlet.py" not in args[2].tb_frame.f_code.co_filename: # Don't display error twice if "greenlet.py" not in args[2].tb_frame.f_code.co_filename: # Don't display error twice
sys.__excepthook__(*args) sys.__excepthook__(*args, **kwargs)
# Ignore notify errors # Ignore notify errors
def handleErrorNotify(*args): def handleErrorNotify(*args, **kwargs):
if args[0].__name__ == "KeyboardInterrupt": err = args[0]
shutdown() if err.__name__ == "KeyboardInterrupt":
if args[0].__name__ != "Notify": shutdown("Keyboard interrupt")
logging.exception("Unhandled exception") elif err.__name__ != "Notify":
sys.__excepthook__(*args) logging.error("Unhandled exception: %s" % [args])
sys.__excepthook__(*args, **kwargs)
if config.debug: # Keep last error for /Debug if config.debug: # Keep last error for /Debug
@ -79,6 +81,12 @@ def handleGreenletError(self, context, type, value, tb):
gevent.hub.Hub.handle_error = handleGreenletError gevent.hub.Hub.handle_error = handleGreenletError
try:
signal.signal(signal.SIGTERM, lambda signum, stack_frame: shutdown("SIGTERM"))
except Exception as err:
logging.debug("Error setting up SIGTERM watcher: %s" % err)
if __name__ == "__main__": if __name__ == "__main__":
import time import time
from gevent import monkey from gevent import monkey

View file

@ -146,7 +146,7 @@ class Peer(object):
self.log("Send request: %s %s %s %s" % (params.get("site", ""), cmd, params.get("inner_path", ""), params.get("location", ""))) self.log("Send request: %s %s %s %s" % (params.get("site", ""), cmd, params.get("inner_path", ""), params.get("location", "")))
for retry in range(1, 4): # Retry 3 times for retry in range(1, 2): # Retry 1 times
try: try:
if not self.connection: if not self.connection:
raise Exception("No connection found") raise Exception("No connection found")

View file

@ -90,7 +90,7 @@ class Site(object):
self.settings = settings self.settings = settings
if "cache" not in settings: if "cache" not in settings:
settings["cache"] = {} settings["cache"] = {}
if "size_files_optional" not in settings: if "size_optional" not in settings:
settings["size_optional"] = 0 settings["size_optional"] = 0
if "optional_downloaded" not in settings: if "optional_downloaded" not in settings:
settings["optional_downloaded"] = 0 settings["optional_downloaded"] = 0

View file

@ -362,7 +362,7 @@ class SiteStorage(object):
if not inner_path: if not inner_path:
return self.directory return self.directory
if ".." in inner_path: if "../" in inner_path:
raise Exception(u"File not allowed: %s" % inner_path) raise Exception(u"File not allowed: %s" % inner_path)
return u"%s/%s" % (self.directory, inner_path) return u"%s/%s" % (self.directory, inner_path)

View file

@ -39,7 +39,7 @@
" files needs to be downloaded": " i file devono essere scaricati", " files needs to be downloaded": " i file devono essere scaricati",
" downloaded": " scaricati", " downloaded": " scaricati",
" download failed": " scaricamento fallito", " download failed": " scaricamento fallito",
"Peers found: ": "Peer trovati: ", "Peers found: ": "Peers trovati: ",
"No peers found": "Nessun peer trovato", "No peers found": "Nessun peer trovato",
"Running out of size limit (": "Superato il limite di spazio (", "Running out of size limit (": "Superato il limite di spazio (",
"Set limit to \" + site_info.next_size_limit + \"MB": "Imposta il limite a \" + site_info.next_size_limit + \"MB", "Set limit to \" + site_info.next_size_limit + \"MB": "Imposta il limite a \" + site_info.next_size_limit + \"MB",

View file

@ -103,7 +103,7 @@ class UiRequest(object):
extra_headers = {"Access-Control-Allow-Origin": "null"} extra_headers = {"Access-Control-Allow-Origin": "null"}
self.sendHeader(content_type=content_type, extra_headers=extra_headers) self.sendHeader(content_type=content_type, extra_headers=extra_headers, noscript=True)
return "" return ""
if path == "/": if path == "/":
@ -246,7 +246,13 @@ class UiRequest(object):
headers["Connection"] = "Keep-Alive" headers["Connection"] = "Keep-Alive"
headers["Keep-Alive"] = "max=25, timeout=30" headers["Keep-Alive"] = "max=25, timeout=30"
headers["X-Frame-Options"] = "SAMEORIGIN" headers["X-Frame-Options"] = "SAMEORIGIN"
if content_type != "text/html" and self.env.get("HTTP_REFERER") and self.isSameOrigin(self.getReferer(), self.getRequestUrl()): is_referer_allowed = False
if self.env.get("HTTP_REFERER"):
if self.isSameOrigin(self.getReferer(), self.getRequestUrl()):
is_referer_allowed = True
elif self.getReferer() == "%s://%s/" % (self.env["wsgi.url_scheme"], self.env["HTTP_HOST"]): # Origin-only referer
is_referer_allowed = True
if content_type != "text/html" and is_referer_allowed:
headers["Access-Control-Allow-Origin"] = "*" # Allow load font files from css headers["Access-Control-Allow-Origin"] = "*" # Allow load font files from css
if noscript: if noscript:
@ -287,9 +293,12 @@ class UiRequest(object):
# Renders a template # Renders a template
def render(self, template_path, *args, **kwargs): def render(self, template_path, *args, **kwargs):
template = open(template_path).read() template = open(template_path).read()
for key, val in kwargs.items(): def renderReplacer(m):
template = template.replace("{%s}" % key, "%s" % val) return "%s" % kwargs.get(m.group(1), "")
return template.encode("utf8")
template_rendered = re.sub("{(.*?)}", renderReplacer, template)
return template_rendered.encode("utf8")
# - Actions - # - Actions -
@ -410,6 +419,9 @@ class UiRequest(object):
file_url = "/" + address + "/" + inner_path file_url = "/" + address + "/" + inner_path
root_url = "/" + address + "/" root_url = "/" + address + "/"
if self.isProxyRequest():
self.server.allowed_ws_origins.add(self.env["HTTP_HOST"])
# Wrapper variable inits # Wrapper variable inits
body_style = "" body_style = ""
meta_tags = "" meta_tags = ""
@ -520,7 +532,7 @@ class UiRequest(object):
if path.endswith("/"): if path.endswith("/"):
path = path + "index.html" path = path + "index.html"
if ".." in path or "./" in path: if "../" in path or "./" in path:
raise SecurityError("Invalid path") raise SecurityError("Invalid path")
match = re.match("/media/(?P<address>[A-Za-z0-9]+[A-Za-z0-9\._-]+)(?P<inner_path>/.*|$)", path) match = re.match("/media/(?P<address>[A-Za-z0-9]+[A-Za-z0-9\._-]+)(?P<inner_path>/.*|$)", path)
@ -704,9 +716,20 @@ class UiRequest(object):
# On websocket connection # On websocket connection
def actionWebsocket(self): def actionWebsocket(self):
ws = self.env.get("wsgi.websocket") ws = self.env.get("wsgi.websocket")
if ws: if ws:
wrapper_key = self.get["wrapper_key"] # Allow only same-origin websocket requests
origin = self.env.get("HTTP_ORIGIN")
host = self.env.get("HTTP_HOST")
# Allow only same-origin websocket requests
if origin:
origin_host = origin.split("://", 1)[-1]
if origin_host != host and origin_host not in self.server.allowed_ws_origins:
ws.send(json.dumps({"error": "Invalid origin: %s" % origin}))
return self.error403("Invalid origin: %s" % origin)
# Find site by wrapper_key # Find site by wrapper_key
wrapper_key = self.get["wrapper_key"]
site = None site = None
for site_check in self.server.sites.values(): for site_check in self.server.sites.values():
if site_check.settings["wrapper_key"] == wrapper_key: if site_check.settings["wrapper_key"] == wrapper_key:
@ -781,30 +804,30 @@ class UiRequest(object):
# Send bad request error # Send bad request error
def error400(self, message=""): def error400(self, message=""):
self.sendHeader(400) self.sendHeader(400, noscript=True)
return self.formatError("Bad Request", message) return self.formatError("Bad Request", message)
# You are not allowed to access this # You are not allowed to access this
def error403(self, message="", details=True): def error403(self, message="", details=True):
self.sendHeader(403) self.sendHeader(403, noscript=True)
self.log.error("Error 403: %s" % message) self.log.error("Error 403: %s" % message)
return self.formatError("Forbidden", message, details=details) return self.formatError("Forbidden", message, details=details)
# Send file not found error # Send file not found error
def error404(self, path=""): def error404(self, path=""):
self.sendHeader(404) self.sendHeader(404, noscript=True)
return self.formatError("Not Found", cgi.escape(path.encode("utf8")), details=False) return self.formatError("Not Found", path.encode("utf8"), details=False)
# Internal server error # Internal server error
def error500(self, message=":("): def error500(self, message=":("):
self.sendHeader(500) self.sendHeader(500, noscript=True)
return self.formatError("Server error", cgi.escape(message)) return self.formatError("Server error", message)
def formatError(self, title, message, details=True): def formatError(self, title, message, details=True):
import sys import sys
import gevent import gevent
if details: if details and config.debug:
details = {key: val for key, val in self.env.items() if hasattr(val, "endswith") and "COOKIE" not in key} details = {key: val for key, val in self.env.items() if hasattr(val, "endswith") and "COOKIE" not in key}
details["version_zeronet"] = "%s r%s" % (config.version, config.rev) details["version_zeronet"] = "%s r%s" % (config.version, config.rev)
details["version_python"] = sys.version details["version_python"] = sys.version
@ -819,10 +842,10 @@ class UiRequest(object):
</style> </style>
<h1>%s</h1> <h1>%s</h1>
<h2>%s</h3> <h2>%s</h3>
<h3>Please <a href="https://github.com/HelloZeroNet/ZeroNet/issues" target="_blank">report it</a> if you think this an error.</h3> <h3>Please <a href="https://github.com/HelloZeroNet/ZeroNet/issues" target="_top">report it</a> if you think this an error.</h3>
<h4>Details:</h4> <h4>Details:</h4>
<pre>%s</pre> <pre>%s</pre>
""" % (title, message, json.dumps(details, indent=4, sort_keys=True)) """ % (title, cgi.escape(message), cgi.escape(json.dumps(details, indent=4, sort_keys=True)))
else: else:
return """ return """
<h1>%s</h1> <h1>%s</h1>

View file

@ -75,6 +75,7 @@ class UiServer:
else: else:
self.allowed_hosts = set([]) self.allowed_hosts = set([])
self.allow_trans_proxy = config.ui_trans_proxy self.allow_trans_proxy = config.ui_trans_proxy
self.allowed_ws_origins = set()
self.wrapper_nonces = [] self.wrapper_nonces = []
self.add_nonces = [] self.add_nonces = []

View file

@ -1120,6 +1120,11 @@ class UiWebsocket(object):
self.response(to, {"error": "Forbidden you cannot set this config key"}) self.response(to, {"error": "Forbidden you cannot set this config key"})
return return
if key == "open_browser":
if value not in ["default_browser", "False"]:
self.response(to, {"error": "Forbidden: Invalid value"})
return
# Remove empty lines from lists # Remove empty lines from lists
if type(value) is list: if type(value) is list:
value = [line for line in value if line] value = [line for line in value if line]

View file

@ -1,5 +1,5 @@
[ req ] [ req ]
prompt = no prompt = yes
default_bits = 2048 default_bits = 2048
default_keyfile = server-key.pem default_keyfile = server-key.pem
distinguished_name = subject distinguished_name = subject
@ -32,8 +32,8 @@ authorityKeyIdentifier = keyid,issuer
basicConstraints = CA:FALSE basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = @alternate_names subjectAltName = @alternate_names
nsComment = "OpenSSL Generated Certificate"
# RFC 5280, Section 4.2.1.12 makes EKU optional # RFC 5280, Section 4.2.1.12 makes EKU optional
# CA/Browser Baseline Requirements, Appendix (B)(3)(G) makes me confused # CA/Browser Baseline Requirements, Appendix (B)(3)(G) makes me confused
@ -46,8 +46,8 @@ subjectKeyIdentifier = hash
basicConstraints = CA:FALSE basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = @alternate_names subjectAltName = @alternate_names
nsComment = "OpenSSL Generated Certificate"
# RFC 5280, Section 4.2.1.12 makes EKU optional # RFC 5280, Section 4.2.1.12 makes EKU optional
# CA/Browser Baseline Requirements, Appendix (B)(3)(G) makes me confused # CA/Browser Baseline Requirements, Appendix (B)(3)(G) makes me confused
@ -55,16 +55,5 @@ nsComment = "OpenSSL Generated Certificate"
[ alternate_names ] [ alternate_names ]
DNS.1 = example.com DNS.1 = $ENV::CN
DNS.2 = www.example.com DNS.2 = www.$ENV::CN
DNS.3 = mail.example.com
DNS.4 = ftp.example.com
# Add these if you need them. But usually you don't want them or
# need them in production. You may need them for development.
# DNS.5 = localhost
# DNS.6 = localhost.localdomain
# DNS.7 = 127.0.0.1
# IPv6 localhost
# DNS.8 = ::1

View file

@ -72,6 +72,13 @@ def getFreeSpace():
return free_space return free_space
def sqlquote(value):
if type(value) is int:
return str(value)
else:
return "'%s'" % value.replace("'", "''")
def shellquote(*args): def shellquote(*args):
if len(args) == 1: if len(args) == 1:
return '"%s"' % args[0].replace('"', "") return '"%s"' % args[0].replace('"', "")

View file

@ -9,6 +9,7 @@ import zeronet
def main(): def main():
if "--open_browser" not in sys.argv:
sys.argv = [sys.argv[0]] + ["--open_browser", "default_browser"] + sys.argv[1:] sys.argv = [sys.argv[0]] + ["--open_browser", "default_browser"] + sys.argv[1:]
zeronet.main() zeronet.main()

View file

@ -11,12 +11,6 @@ def main():
main = None main = None
try: try:
import signal
try:
signal.signal(signal.SIGTERM, lambda signum, stack_frame: sys.exit(0))
except Exception as err:
print("Error setting up SIGTERM watcher: %s" % err)
app_dir = os.path.dirname(os.path.abspath(__file__)) app_dir = os.path.dirname(os.path.abspath(__file__))
os.chdir(app_dir) # Change working dir to zeronet.py dir os.chdir(app_dir) # Change working dir to zeronet.py dir
sys.path.insert(0, os.path.join(app_dir, "src/lib")) # External liblary directory sys.path.insert(0, os.path.join(app_dir, "src/lib")) # External liblary directory