Skip to main content

Engelbert Tejeda

Engelbert Tejeda's Public Library

about 2 hours ago

"import fnmatch
import os
import os.path
import re

includes = ['*.doc', '*.odt'] # for files only
excludes = ['/home/paulo-freitas/Documents'] # for dirs and files

# transform glob patterns to regular expressions
includes = r'|'.join([fnmatch.translate(x) for x in includes])
excludes = r'|'.join([fnmatch.translate(x) for x in excludes]) or r'$.'

for root, dirs, files in os.walk('/home/paulo-freitas'):

# exclude dirs
dirs[:] = [os.path.join(root, d) for d in dirs]
dirs[:] = [d for d in dirs if not re.match(excludes, d)]

# exclude/include files
files = [os.path.join(root, f) for f in files]
files = [f for f in files if not re.match(excludes, f)]
files = [f for f in files if re.match(includes, f)]

for fname in files:
print fname"

about 2 hours ago

kwd:{pip2pi builds a PyPI-compatible package repository from pip requirements
kwd:{pip index, repository, packages, http, repo

about 6 hours ago

"I have a few files on the downloads page on my bitbucket repository. Is there a way I can download these files from the command line using the SSH key I've stored with my account? Or what's the easiest way to do it?"
kwd:{curl --user username -L -O https://bitbucket.org/account/repo/downloads/myfile.tar.gz

about 7 hours ago

"Some useful socat commands
To link serial port ttyS0 to another serial port:

socat /dev/ttyS0,raw,echo=0,crnl /dev/ttyS1,raw,echo=0,crnl

To get time from time server:

socat TCP:time.nist.gov:13 -

To forward local http port to remote http port:

socat TCP-LISTEN:80,fork TCP:www.domain.org:80

To forward terminal to the serial port COM1:

socat READLINE,history=$HOME/.cmd_history /dev/ttyS0,raw,echo=0,crnl

Simple file-transfer:

On the server-side: socat TCP-LISTEN:port filename
To send file fro the server: socat TCP:hostname:port filename

socat - TCP4:www.domain.org:80

Transfers data between STDIO (-) and a TCP4 connection to port 80 of host www.domain.org. This example results in an interactive connection similar to telnet or netcat. The stdin terminal parameters are not changed, so you may close the relay with ^D or abort it with ^C.

socat -d -d READLINE,history=$HOME/.http_history \
TCP4:www.domain.org:www,crnl

This is similar to the previous example, but you can edit the current line in a bash like manner (READLINE) and use the history file .http_history; socat prints messages about progress (-d -d). The port is specified by service name (www), and correct network line termination characters (crnl) instead of NL are used.

socat TCP4-LISTEN:www TCP4:www.domain.org:www

Installs a simple TCP port forwarder. With TCP4-LISTEN it listens on local port "www" until a connection comes in, accepts it, then connects to the remote host (TCP4) and starts data transfer. It will not accept a second connection.


socat -d -d -lmlocal2 \
TCP4-LISTEN:80,bind=myaddr1,su=nobody,fork,range=10.0.0.0/8,reuseaddr \
TCP4:www.domain.org:80,bind=myaddr2

TCP port forwarder, each side bound to another local IP address (bind). This example handles an almost arbitrary number of parallel or consecutive connections by forking a new process after each accept(). It provides a little security by sudoing to user nobody after forking; it only permits connections from the private 10 network (range); due to reuseaddr, it allows immediate restart after master processes termination, even if some child sockets are not completely shut down. With -lmlocal2, socat logs to stderr until successfully reaching the accept loop. Further logging is directed to syslog with facility local2.


socat TCP4-LISTEN:5555,fork,tcpwrap=script \
EXEC:/bin/myscript,chroot=/home/sandbox,su-d=sandbox,pty,stderr

A simple server that accepts connections (TCP4-LISTEN) and forks a new child process for each connection; every child acts as single relay. The client must match the rules for daemon process name "script" in /etc/hosts.allow and /etc/hosts.deny, otherwise it is refused access (see "man 5 hosts_access"). For EXECuting the program, the child process chroots to /home/sandbox, sus to user sandbox, and then starts the program /home/sandbox/bin/myscript. Socat and myscript communicate via a pseudo tty (pty); myscripts stderr is redirected to stdout, so its error messages are transferred via socat to the connected client.

socat EXEC:"mail.sh target@domain.com",fdin=3,fdout=4 \
TCP4:mail.relay.org:25,crnl,bind=alias1.server.org,mss=512

mail.sh is a shell script, distributed with socat, that implements a simple SMTP client. It is programmed to "speak" SMTP on its FDs 3 (in) and 4 (out). The fdin and fdout options tell socat to use these FDs for communication with the program. Because mail.sh inherits stdin and stdout while socat does not use them, the script can read a mail body from stdin. Socat makes alias1 your local source address (bind), cares for correct network line termination (crnl) and sends at most 512 data bytes per packet (mss).


socat - /dev/ttyS0,raw,echo=0,crnl

Opens an interactive connection via the serial line, e.g. for talking with a modem. raw and echo set ttyS0's terminal parameters to practicable values, crnl converts to correct newline characters. Consider using READLINE instead of `-'.


socat UNIX-LISTEN:/tmp/.X11-unix/X1,fork \
SOCKS4:host.victim.org:127.0.0.1:6000,socksuser=nobody,sourceport=20

With UNIX-LISTEN, socat opens a listening UNIX domain socket /tmp/.X11-unix/X1. This path corresponds to local XWindow display :1 on your machine, so XWindow client connections to DISPLAY=:1 are accepted. Socat then speaks with the SOCKS4 server host.victim.org that might permit sourceport 20 based connections due to an FTP related weakness in its static IP filters. Socat pretends to be invoked by socksuser nobody, and requests to be connected to loopback port 6000 (only weak sockd configurations will allow this). So we get a connection to the victims XWindow server and, if it does not require MIT cookies or Kerberos authentication, we can start work. Please note that there can only be one connection at a time, because TCP can establish only one session with a given set of addresses and ports.


socat -u /tmp/readdata,seek-end=0,ignoreeof -


This is an example for unidirectional data transfer (-u). Socat transfers data from file /tmp/readdata (implicit address GOPEN), starting at its current end (seek-end=0 lets socat start reading at current end of file; use seek=0 or no seek option to first read the existing data) in a "tail -f" like mode (ignoreeof). The "file" might also be a listening UNIX domain socket (do not use a seek option then).

(sleep 5; echo PASSWORD; sleep 5; echo ls; sleep 1) |
socat - EXEC:'ssh -l user server',pty,setsid,ctty

EXECutes an ssh session to server. Uses a pty for communication between socat and ssh, makes it ssh's controlling tty (ctty), and makes this pty the owner of a new process group (setsid), so ssh accepts the password from socat.


socat -u TCP4-LISTEN:3334,reuseaddr,fork \
OPEN:/tmp/in.log,creat,append


Implements a simple network based message collector. For each client connecting to port 3334, a new child process is generated (option fork). All data sent by the clients are appended to the file /tmp/in.log. If the file does not exist, socat creats it. Option reuseaddr allows immediate restart of the server process.


socat READLINE,noecho='[Pp]assword:' EXEC:'ftp ftp.server.com',pty,setsid,ctty


Wraps a command line history (READLINE) around the EXECuted ftp client utility. This allows editing and reuse of FTP commands for relatively comfortable browsing through the ftp directory hierarchy. The password is echoed! pty is required to have ftp issue a prompt. Nevertheless, there may occur some confusion with the password and FTP prompts.


socat PTY,link=$HOME/dev/vmodem0,raw,echo=0,waitslave exec:'


Generates a pseudo terminal device (PTY) on the client that can be reached under the symbolic link $HOME/dev/vmodem0. An application that expects a serial line or modem can be configured to use $HOME/dev/vmodem0; its traffic will be directed to a modemserver via ssh where another socat instance links it with /dev/ttyS0.


socat TCP4-LISTEN:2022,reuseaddr,fork \
PROXY:proxy:www.domain.org:22,proxyport=3128,proxyauth=user:pass

starts a forwarder that accepts connections on port 2022, and directs them through the proxy daemon listening on port 3128 (proxyport) on host proxy, using the CONNECT method, where they are authenticated as "user" with "pass" (proxyauth). The proxy should establish connections to host www.domain.org on port 22 then.


echo |socat -u - file:/tmp/bigfile,create,largefile,seek=100000000000


creates a 100GB sparse file; this requires a file system type that supports this (ext2, ext3, reiserfs, jfs; not minix, vfat). The operation of writing 1 byte might take long (reiserfs: some minutes; ext2: "no" time), and the resulting file can consume some disk space with just its inodes (reiserfs: 2MB; ext2:16KB).


socat tcp-l:7777,reuseaddr,fork system:filan -i 0 -s >&2,nofork

listens for incoming TCP connections on port 7777. For each accepted connection, invokes a shell. This shell has its stdin and stdout directly connected to the TCP socket (nofork). The shell starts filan and lets it print the socket addresses to stderr (your terminal window).

echo -e

functions as primitive binary editor: it writes the 4 bytes 000 014 000 000 to the executable /usr/bin/squid at offset 0x00074420 (this is a real world patch to make the squid executable from Cygwin run under Windows, actual per May 2004).


socat - tcp:www.blackhat.org:31337,readbytes=1000

connect to an unknown service and prevent being flooded."

about 7 hours ago

kwd:{read excel file from web server
"You can pass your socket to ExcelFile:

>>> import pandas as pd
>>> import urllib2
>>> link = 'http://www.econ.yale.edu/~shiller/data/chapt26.xls'
>>> socket = urllib2.urlopen(link)
>>> xd = pd.ExcelFile(socket)
NOTE *** Ignoring non-worksheet data named u'PDVPlot' (type 0x02 = Chart)
NOTE *** Ignoring non-worksheet data named u'ConsumptionPlot' (type 0x02 = Chart)
>>> xd.sheet_names
[u'Data', u'Consumption', u'Calculations']
>>> df = xd.parse(xd.sheet_names[-1], header=None)
>>> df
0 1 2 3 4
0 Average Real Interest Rate: NaN NaN NaN 1.028826
1 Geometric Average Stock Return: NaN NaN NaN 0.065533
2 exp(geo. Avg. return) NaN NaN NaN 0.067728
3 Geometric Average Dividend Growth NaN NaN NaN 0.012025"

about 8 hours ago

"DB Browser for SQLite
The Official home of the DB Browser for SQLite"

about 10 hours ago

gls*working with sqlite3 on osx

about 11 hours ago

gls*python recurse directories
kwd:{"import os
import sys

walk_dir = sys.argv[1]

print('walk_dir = ' + walk_dir)

# If your current working directory may change during script execution, it's recommended to
# immediately convert program arguments to an absolute path. Then the variable root below will
# be an absolute path as well. Example:
# walk_dir = os.path.abspath(walk_dir)
print('walk_dir (absolute) = ' + os.path.abspath(walk_dir))

for root, subdirs, files in os.walk(walk_dir):
print('--\nroot = ' + root)
list_file_path = os.path.join(root, 'my-directory-list.txt')
print('list_file_path = ' + list_file_path)

with open(list_file_path, 'wb') as list_file:
for subdir in subdirs:
print('\t- subdirectory ' + subdir)

for filename in files:
file_path = os.path.join(root, filename)

print('\t- file %s (full path: %s)' % (filename, file_path))

with open(file_path, 'rb') as f:
f_content = f.read()
list_file.write(('The file %s contains:\n' % filename).encode('utf-8'))
list_file.write(f_content)
list_file.write(b'\n')"

about 14 hours ago

gls*"socat" "br0"
kwd:{I can also use Pcap-over-IP to capture network traffic from a remote PC or device. I can, for example use tcpdump to sniff traffic on the external interface of my Linux-based firewall and push it to an analyst station like this:

# tcpdump -i eth1 -s 0 -U -w - | nc 192.168.1.20 57012
I can also perform remote WiFi sniffing with dumpcap or tcpdump from a Linux machine and send the sniffed packets to NetworkMiner with netcat like this:

# iwconfig wlan0 mode monitor
# iwconfig wlan0 channel 4
# dumpcap -i wlan0 -P -w - | nc 192.168.1.20 57012

about 15 hours ago

gls*"eval _time=relative_time" "utc"
"So a different note here, ( Reference ) date_hour is only present if the timestamp exists and is extracted from the raw event itself. It's also in the timezone of the log and not the timezone of you the splunk user (_time is the UTC timestamp, and is by default displayed in your timezone)... which could be important if you are combining logs from different timezones or across days. You could also use |eval _time=relative_time(_time,"@h"), or |bin _time span=1h or |eval hour=strftime(_time, "%H") for getting a field by hour based on the _time field."

1 - 20 of 22495 Next › Last »
20 items/page

Highlighter, Sticky notes, Tagging, Groups and Network: integrated suite dramatically boosting research productivity. Learn more »

Join Diigo