Check out my first novel, midnight's simulacra!
Hackery: Difference between revisions
From dankwiki
(→Various open source contributions (very incomplete): fix ethereal (wireshark) link) |
|||
(26 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
==Open source | ==Open source detrital TODOs== | ||
* speed up update-mandb | |||
* fix up ext3grep to work on ext4 filesystems | * fix up ext3grep to work on ext4 filesystems | ||
* Implement handling of [[C99]] [http://gcc.gnu.org/onlinedocs/gcc/Variadic-Macros.html variadic macros] in [http://www.splint.org/ splint] (see [http://www.advogato.org/person/muks/diary.html/start=0 this], [http://www.buzztard.org/index.php/Splint this] and [http://www.splint.org/faq.html#quest18b of course this]) | * Implement handling of [[C99]] [http://gcc.gnu.org/onlinedocs/gcc/Variadic-Macros.html variadic macros] in [http://www.splint.org/ splint] (see [http://www.advogato.org/person/muks/diary.html/start=0 this], [http://www.buzztard.org/index.php/Splint this] and [http://www.splint.org/faq.html#quest18b of course this]) | ||
Line 8: | Line 9: | ||
* [[mpd]] ought use filesystem change notification events to trigger database changes, not periodic or manual rescans | * [[mpd]] ought use filesystem change notification events to trigger database changes, not periodic or manual rescans | ||
* [[valgrind]] is missing some obscure ioctl's, including [[ethtool|ethtool's]] | * [[valgrind]] is missing some obscure ioctl's, including [[ethtool|ethtool's]] | ||
* canScan | |||
* systemd Restart-with-oneshot fix | |||
* use [[Outcurses]] with usbtop for ncurses mode | |||
* fix bug in usbtop where disappeared devices remain forever | |||
* systemd system-wide service for pulseaudio (requested on freedesktop.org wiki!) | |||
* cuda fft for various SDR | |||
* step-cli -- update debian stuff, make it bizzuild | |||
* ubertooth-dfu without `-d` just exits cleanly, doing nothing | |||
* neat fpga project -- bluetooth 5 (2mbit ble channel) 40x channel sniffer | |||
===Various open source contributions (very incomplete)=== | ===Various open source contributions (very incomplete)=== | ||
* doctest: | |||
** [https://github.com/onqtam/doctest/pull/403 properly align] summary table even for large values | |||
* Kitty: | |||
** [https://github.com/kovidgoyal/kitty/pull/3105 fixed] a bug in sextant drawing | |||
* [[CMake]]: | |||
** [https://gitlab.kitware.com/cmake/cmake/merge_requests/3845 fixed] up the Curses module | |||
* [https://github.com/OpenShot/libopenshot OpenShot]: | |||
** [https://github.com/OpenShot/libopenshot/commit/4a1d133da85e529b158f9a34518c41feb150d71c fixed] a compiler warning | |||
* libqmi: | |||
** added support for [https://gitlab.freedesktop.org/mobile-broadband/libqmi/issues/10 device specification using symlinks] | |||
* Alpine Linux: | |||
** added parsechangelog tools to dpkg | |||
** packaged capnproto and nlohmann-json | |||
* Compiz: | |||
** fixed [https://github.com/compiz-reloaded/compiz/commit/8884ea41b72a96ecc8bf55029033af48d3d3fb97 test for decorator specification] | |||
* iperf2: | |||
** [https://sourceforge.net/p/iperf2/discussion/general/thread/b27f2bc6 Fixed] bad delete leading to memory corruption | |||
* [[Ncurses]]: | |||
** [http://comments.gmane.org/gmane.comp.lib.ncurses.bugs/4910 Fixed] COLOR_PAIR() and PAIR_NUMBER() macros | |||
* [[Valgrind]]: | * [[Valgrind]]: | ||
** Added support for [https://bugs.kde.org/show_bug.cgi?id=302827 CDROM_GET_CAPABILITY] <tt>ioctl</tt> | ** Added support for [https://bugs.kde.org/show_bug.cgi?id=302827 CDROM_GET_CAPABILITY] <tt>ioctl</tt> | ||
** Added support for [https://bugs.kde.org/show_bug.cgi?id=410556 BLKID] <tt>ioctls</tt> | |||
* libblkid (util-linux) | * libblkid (util-linux) | ||
** [https://github.com/karelzak/util-linux/commit/ffab21e12846dd9b9403c881721e415493805bd1 Fix #1] | ** [https://github.com/karelzak/util-linux/commit/ffab21e12846dd9b9403c881721e415493805bd1 Fix #1] | ||
Line 18: | Line 48: | ||
** [https://github.com/karelzak/util-linux/commit/28a47f13d9c33c97279742d83687bff4d79922f4 Fix #4] | ** [https://github.com/karelzak/util-linux/commit/28a47f13d9c33c97279742d83687bff4d79922f4 Fix #4] | ||
* [http://www.wireshark.org/about.html Wireshark]: | * [http://www.wireshark.org/about.html Wireshark]: | ||
** [ | ** [https://www.wireshark.org/lists/ethereal-dev/200402/msg00512.html Fixed handling] of IPv4 fragmentation bits. | ||
* [http://www.linuxfoundation.org/collaborate/workgroups/networking/iproute2 iproute2] | * [http://www.linuxfoundation.org/collaborate/workgroups/networking/iproute2 iproute2] | ||
** Many [http://git.kernel.org/?p=linux/kernel/git/shemminger/iproute2.git;a=blobdiff;f=man/man8/ip.8;h=0f9f454737c5a5977026752321d620d7bee79158;hp=68887bacff00e6e03f4e035a5935f557aa79bd83;hb=db4a7f198b6193a1e584c3b6647d92bb6c66fb52;hpb=14a1c164d12e32415acc44ef566fcf52ff4dd113 man page] fixes | ** Many [http://git.kernel.org/?p=linux/kernel/git/shemminger/iproute2.git;a=blobdiff;f=man/man8/ip.8;h=0f9f454737c5a5977026752321d620d7bee79158;hp=68887bacff00e6e03f4e035a5935f557aa79bd83;hb=db4a7f198b6193a1e584c3b6647d92bb6c66fb52;hpb=14a1c164d12e32415acc44ef566fcf52ff4dd113 man page] fixes | ||
* [[Linux APIs|Linux kernel]]: | * [[Linux APIs|Linux kernel]]: | ||
** [https://patchwork.kernel.org/patch/11039443/ Whitelisted] the [[Lenovo]] T580's synaptics touchpad for SMBus | |||
** [http://lkml.org/lkml/2010/5/4/6 Added support] for Model-30 [[Nehalem]] processors in [[Oprofile]] ([http://marc.info/?l=linux-kernel&m=127294830417492&w=2 more discussion] on oprofile-users) | ** [http://lkml.org/lkml/2010/5/4/6 Added support] for Model-30 [[Nehalem]] processors in [[Oprofile]] ([http://marc.info/?l=linux-kernel&m=127294830417492&w=2 more discussion] on oprofile-users) | ||
** [http://lkml.indiana.edu/hypermail/linux/kernel/0906.3/02414.html Addressed issues] in [[pages|HugeTLBFS]]'s API. | ** [http://lkml.indiana.edu/hypermail/linux/kernel/0906.3/02414.html Addressed issues] in [[pages|HugeTLBFS]]'s API. | ||
** [http://lkml.indiana.edu/hypermail/linux/net/0301.1/0037.html Addressed] an issue in IPv4 address/route removal. | ** [http://lkml.indiana.edu/hypermail/linux/net/0301.1/0037.html Addressed] an issue in IPv4 address/route removal. | ||
** Worked out some kinks in Matrox G400 framebuffer and SiS-5591 IDE drivers. | ** Worked out some kinks in Matrox G400 framebuffer and SiS-5591 IDE drivers. | ||
** [ | ** [https://fossies.org/linux/privat/old/ngpt-2.2.1.tar.gz:a/ngpt-2.2.1/THANKS Next Generation Posix Threading] work + maintenance | ||
* [http://sourceforge.net/projects/strace/ strace]: | * [http://sourceforge.net/projects/strace/ strace]: | ||
** [http://www.mail-archive.com/strace-devel@lists.sourceforge.net/msg00614.html Addressed issues] in various architectures' [[Linux APIs|sendfile(2)]] support. | ** [http://www.mail-archive.com/strace-devel@lists.sourceforge.net/msg00614.html Addressed issues] in various architectures' [[Linux APIs|sendfile(2)]] support. | ||
Line 38: | Line 69: | ||
* iw | * iw | ||
** [http://thread.gmane.org/gmane.linux.kernel.wireless.general/82070 Correct unsigned] ints used to hold signed data | ** [http://thread.gmane.org/gmane.linux.kernel.wireless.general/82070 Correct unsigned] ints used to hold signed data | ||
* gnome-session | |||
** [https://bugzilla.gnome.org/show_bug.cgi?id=689009 Fixed] gnome-session-properties man page | |||
* Snort | * Snort | ||
** Various [http://copilotco.com/mail-archives/snort-users.2003/msg06337.html signature documentation] | ** Various [http://copilotco.com/mail-archives/snort-users.2003/msg06337.html signature documentation] | ||
Line 55: | Line 88: | ||
* [[omphalos]]: multi-pronged network discovery | * [[omphalos]]: multi-pronged network discovery | ||
* [[growlight]]: multifaceted disk/adapter tool and system installer for [[SprezzOS]] | * [[growlight]]: multifaceted disk/adapter tool and system installer for [[SprezzOS]] | ||
* [[Xcurses]]: a SYSV [[ncurses|curses]] implementation making direct use of [[X]]-graphics primitives | |||
* [[Outcurses]]: a high-level UI library atop [[ncurses]] | |||
[[CATEGORY: Networking]] | [[CATEGORY: Networking]] | ||
* Various projects on [http://github.com/dankamongmen GitHub] which I've not yet documented here... | * Various projects on [http://github.com/dankamongmen GitHub] which I've not yet documented here... | ||
Line 91: | Line 126: | ||
** Over time, confidence in matching both recurring and new traffic increases | ** Over time, confidence in matching both recurring and new traffic increases | ||
* Uses [[Research ideas#Parvenu|Parvenu]] as a pattern-matching helper | * Uses [[Research ideas#Parvenu|Parvenu]] as a pattern-matching helper | ||
* Forms an analysis engine for [[Omphalos]] | |||
===liburine=== | ===liburine=== | ||
Line 107: | Line 143: | ||
** "[[TCP]] 1323 high-performance extensions are being used in 24% of the connections" | ** "[[TCP]] 1323 high-performance extensions are being used in 24% of the connections" | ||
** "Fragmentation is resulting from port 7000's large UDP emissions, can we tune it this way..." | ** "Fragmentation is resulting from port 7000's large UDP emissions, can we tune it this way..." | ||
** "[http://www.youtube.com/watch?v=i6w9YHfabUU Make an incision, Doctor Limpf.] I'm going to massage the heart." | |||
==Compilation/binaries== | ==Compilation/binaries== |
Latest revision as of 21:44, 5 May 2023
Open source detrital TODOs
- speed up update-mandb
- fix up ext3grep to work on ext4 filesystems
- Implement handling of C99 variadic macros in splint (see this, this and of course this)
- Review cryptographic implementations in pidgin -- last time I looked (back in the bad ol' gaim days), its usage of OpenSSL was all FUBAR
- Subversion's --xml and --ignore-externals options couldn't be used together properly in svn status, at some point. Investigate, rectify.
- Various Debian-related things
- "taking OpenSSL's RAND_bytes() (which uses /dev/urandom, EGD, or a pregenerated seed file) function and stirring that into a target buffer of arbitrary length, such that use of actual high-quality entropy bits is constant (probably a read of several words each time a thread calls into (threadsafe from the start, no *_r() crap!) FAUXRAND_bytes() for the first time) has been on my plate since...well, since just now." (mail to Dr. Richard Vuduc, 2009-09-19)
- mpd ought use filesystem change notification events to trigger database changes, not periodic or manual rescans
- valgrind is missing some obscure ioctl's, including ethtool's
- canScan
- systemd Restart-with-oneshot fix
- use Outcurses with usbtop for ncurses mode
- fix bug in usbtop where disappeared devices remain forever
- systemd system-wide service for pulseaudio (requested on freedesktop.org wiki!)
- cuda fft for various SDR
- step-cli -- update debian stuff, make it bizzuild
- ubertooth-dfu without `-d` just exits cleanly, doing nothing
- neat fpga project -- bluetooth 5 (2mbit ble channel) 40x channel sniffer
Various open source contributions (very incomplete)
- doctest:
- properly align summary table even for large values
- Kitty:
- fixed a bug in sextant drawing
- CMake:
- fixed up the Curses module
- OpenShot:
- fixed a compiler warning
- libqmi:
- added support for device specification using symlinks
- Alpine Linux:
- added parsechangelog tools to dpkg
- packaged capnproto and nlohmann-json
- Compiz:
- iperf2:
- Fixed bad delete leading to memory corruption
- Ncurses:
- Fixed COLOR_PAIR() and PAIR_NUMBER() macros
- Valgrind:
- Added support for CDROM_GET_CAPABILITY ioctl
- Added support for BLKID ioctls
- libblkid (util-linux)
- Wireshark:
- Fixed handling of IPv4 fragmentation bits.
- iproute2
- Many man page fixes
- Linux kernel:
- Whitelisted the Lenovo T580's synaptics touchpad for SMBus
- Added support for Model-30 Nehalem processors in Oprofile (more discussion on oprofile-users)
- Addressed issues in HugeTLBFS's API.
- Addressed an issue in IPv4 address/route removal.
- Worked out some kinks in Matrox G400 framebuffer and SiS-5591 IDE drivers.
- Next Generation Posix Threading work + maintenance
- strace:
- Addressed issues in various architectures' sendfile(2) support.
- x86info:
- DynamoRIO:
- avant-wireless
- An AWN applet which tracks wireless connection properties, and spawns wpa_gui
- iw
- Correct unsigned ints used to hold signed data
- gnome-session
- Fixed gnome-session-properties man page
- Snort
- Various signature documentation
Projects with their own pages
- ptracer: Quick-n-dirty instruction trace generation tool
- libdank: Long-term personal collection of routines / application frameworks with a low-level feel
- TANGE: Terminal Application (Next-Generation Emulation)
- ctxdiff: Context-sensitive fractal fuzzy diffing (not at all like shingleprinting)
- xsh: Exactly what it sounds like!
- makelint, which immediately suggests? lintmake
- libtorque: Multithreaded event handler for UNIX on manycore NUMA
- daytripper: Binary translation to take advantage of Intel's Loop Stream Detector
- CUBAR: collection of tools for testing CUDA's security model
- libcudest: open-source implementation of the CUDA userspace
- LRUmap: O(1) LRU for massive numbers of sets
- omphalos: multi-pronged network discovery
- growlight: multifaceted disk/adapter tool and system installer for SprezzOS
- Xcurses: a SYSV curses implementation making direct use of X-graphics primitives
- Outcurses: a high-level UI library atop ncurses
- Various projects on GitHub which I've not yet documented here...
Core stuff
Parvenu
- One string-matching automaton to Rule them All! It must handle:
- Thousands (millions?) of patterns concurrently
- Initially targeting GigE wire speeds
- UTF-8 (at a minimum) and various transcodings
- Small-memory (embedded) environments
- Most elements of regular expressions (definitely all the syntactic sugar)
- Uses libblaze as a memory management helper
libblaze
- Use cpuid to select and dlopen(2) a processor-specific set of low-level routines
- Observe the details within Ulrich Drepper's fine document!
- Expose cpuid and SMP details
- Expose algorithms for memory management / layout tuned to cache/DRAM parameters (detected with SPD)
- Material largely inspired by Warren's Hacker's Delight.
- And to a lesser, far less rigorous extent, Kaspersky's Effective Memory Usage.
- Might have already been superseded by liboil. That damn open source community moves fast!
Networking
Zetetic
- Network traffic analysis (ala Wireshark or tcpdump) with strong covert channel detection
- Passive network modeling (protocols, services, hosts, versions) initially
- To be paired with vulnerability correlation, active probing, histories (ala SourceFire RNA?)
- Higher layers do not restrict the inductive analysis of lower layers, but influence the deduction
- Automata-based analysis of content builds up possibility space of what it CAN be
- Probability multipliers (products of series) paired with non-deterministic automata
- Expert knowledge-based reduction of possibility space adds input as to what it OUGHT be
- Neural net-based learning with state determines what it IS
- Over time, confidence in matching both recurring and new traffic increases
- Uses Parvenu as a pattern-matching helper
- Forms an analysis engine for Omphalos
liburine
- The inverse of Zetetic. Take a leak (alternatively: "You're in")!
- Use arbitrary combinations of protocols (and fuzzing/embedding thereof) to find channels
- Goals: if even a bit of controllable information can be used as a channel
- Goals: Zetetic should be able to find us, but nothing of lesser power!
- The ultimate assistant for: whistleblowers, samizdat publishers, hax0rs stuck in airports, Iranians
drbenway
- Checks sysctls, netstat, /proc, etc for network settings. Makes extensive sensible recommendations.
- Is device polling / NAPI in use? Do firewall rules disallow PMTU discovery?
- Is TCP FRTO being used in the presence of wireless links? etc
- Analyzes pcaps or raw sockets in situ, with filters, performing detailed diagnostics ie
- "TCP 1323 high-performance extensions are being used in 24% of the connections"
- "Fragmentation is resulting from port 7000's large UDP emissions, can we tune it this way..."
- "Make an incision, Doctor Limpf. I'm going to massage the heart."
Compilation/binaries
gcc stuff
- Work on RABLET
- It'd be interesting to take GIMPLE and apply it to malware analysis
- Work on finishing out ISO C99 support
- Work on Coding Rule Checking (GGCC)
debugging / reverse engineering tools
- getgetopt -- discover a program's command line parameters via object analysis
- maybe something to test combinations of command line parameters, as well?
- getgccopt -- discover the compiler options used to build an object. difficult!
- arrlimiter -- run object code with various rlimit restrictions to test failure paths
- ploom -- pthreads debugging from process (tracing framework, object lookup + pthread knowledge, contention measures, etc)