About charleswyble

CTO and cofounder of the FNF. Heading up all things tech (back end and product development)

Free Network Definition / Freedoms 2.0 – restoring consistency


We’ve received quite a bit of feedback regarding the “four freedoms”
and FND being inconsistent. The various versions became inconsistent due
to a prolonged, convoluted and disjointed editorial process that spanned
multiple continents, months of time and almost 20 people.

Our apologies for any confusion this has caused.

At this time the wiki has been updated to reflect the
version as defined at top of NCL. They are:

* You have the right to join the network, and the responsibility to
extend this set of rights to anyone, under these same conditions.
* You have the right to understand the network, to know its
components, and to spread knowledge of its elements and principles.
* You have the right to offer services and content to the network on
your own terms.
* You have the freedom to use the network for any purpose as long as
you don’t harm the operation of the network itself, the rights of other
participants, or the principles of neutrality that allow contents and
services to flow without deliberate interference.

Please find it online here:


Thank you.

Keeping tabs on FNF


Going forward, each Sunday at 7:30PM CST the leadership team will have
it’s weekly conference call. This call is open to anyone who wishes to
participate. The call will be (ideally) 30 minutes in length, and is an
opportunity for quick report backs, general strategy updates etc.

I’ll be setting up a cron job to auto e-mail out a reminder every Sunday
morning with the call details.

If you would like to create a reminder in your calendar, here are all
the details:

When? Sundays at 7:30PM CST to 8:00PM CST
Where? dial in: 559-546-1000 with access code: 431041#

Thanks all.

Network Commons License v0.3 – please give us feedback!


It is my great pleasure to take the wraps off of the NCL v0.3.


This is the latest and greatest refinement of the license which we hope
will govern free networks. This is the plain English version that FNF
would like to get to lawyers on 10/30 for starting the process of
turning into a license. Please comment here with any and all feedback
and distribute as widely as possible for exposure and commentary.

Thank you.

Please find it online here:


To comment on the NCL, please join our discussion mailing list:


FNF Austin Update

Had a crazy productive weekend…

1) Did a huge amount of work on the lab. Lots of cabling, labeling, documenting.
2) Deployed a large format printer
3) obtained a Nortel sip phone
4) obtained two OLPC laptops
5) swapped one of the laptops for a Cisco 7960 SIP phone
6) Cleared up a bunch of things that had cluttered up the lab workspaces. Then immediately filled them up again with more stuff from downstairs.
7) Deployed, powered up access points throughout the house for continuous network testing.
8) wrote a post for /r/homelab with lots of lab details.
9) Read over the engineering study. Great work by Isaac. I will be making some edits to it for consistency and brand reference.

Several small tasks left, I have Tuesday/Wednesday evening dedicated to wrapping things up so Isaac can share the lab with summit folks and hopefully they will use it.

I’m giving a technical FNF FreeNetworks/FreedomStack presentation to the Lopsa Austin chapter tonight.


FNF Austin Update

This evening was spent on getting more of the lab online after my move.


1) Cleaned up power cabling for my production kit.

2) Wired up ubiquiti ip camera and two b/w analog cameras. This is for generating iptv feeds for a future part of FreedomStack called FreedomVid (neighbors YouTube).

3) Wired up two cisco aironet access points in their final location.

4) Deployed power cabling for all the lab cisco switches/routers and FreedomLink test systems.

All cabling is managed (as well as I can manage power bricks and computer power cords without a rack that is!). Im happy with the layout and configuration. Power in back, data in front. All devices are now on managed PDU porte.

The setup is pretty permanent, as I don’t plan to move for a very long time, and I’m free of the space and sharing constraints of my apartment.

5) Organized my toolbag, it is much more oriented towards more field/tower type work now.

Tomorrow/Thursday I will be bringing the cyclades online, finishing up data/power cabling for all lab kit, bringing up wireless spectrum/network/IDS monitor, upgrading to pfsense 2.1 release, documenting everything and posting up pics

I hope to have the lab infrastrucfure and documentation (pics, wiki page, racktables) fully finished Thursday night by about 11pm.

Friday night will be getting the access points flashed with QMP.

Saturday will be the mail server rebuild.

Next week will be full out hax on AutoNOC.

This will put us in a great place for the summit in Berlin. It will show we can get things done. Lets keep at it folks!

FNF Lab Update

Hey all,

I’ve been pretty busy with settling in to the new place the last couple of weeks, and haven’t had much to report on the FNF front. I’ve mostly wrapped all that up , and should be finishing up over the next couple weeks.

I got quite a bit done this weekend on the FNF front.

1) Deployed 8 access points throughout the house, both upstairs and downstairs. This will let us have a live dev network to develop AutoNOC against. Floor plans, modeling etc are all underway.

2) Got a qmp dev environment setup on the lab server and built images for the various access points.

3) Deployed observium and have it instrumenting (via snmp) the house production network (pfsense router, cisco 3550 core switch, cisco 1231 G ap, netgear wndr3800 G/N ap). In particular I’m interested in seeing what happens to small SOHO networks near meshes. Co existence and transition are vital parts of how we can
support folks doing mesh deployment.

4) Synced with Isaac on all the things. Its great to be coming back online.

5) FreedomBot MARK1 is well underway. Lots of pics were taken of the progress. Post up soon.

6) Presenting to the Austin LOPSA chapter on 9/23 (week from today) on FNF. It will be a very busy week/weekend for me to have the lab online by the presentation.

On a personal note, I’ve cut out all TV watching. Its done wonders for my productivity!

Tonight I will be wrapping up some non FNF things and doing some final lab wiring, tomorrow I will be flashing all the radios. I will post another update (including task warrior bits and links to pics/wiki pages/AutoNOC alpha) on Wednesday or Thursday when I’ve got something more substantial to report.

How this blog gets delivered to our visitors

Lately I’ve been improving our technical operations and infrastructure related documentation, in preparation for a public release. I wanted to give folks a peek into one of the most mature aspects of that documentation, and give a “look behind the curtain” at how we do web related operations at the FNF.

Originally this site (and our numerous sub domains) were served from a cPanel virtual machine running on a USB hard drive directly attached to the server.  We’ve now moved away from that, to the infrastructure defined here:


As you can see, many moving parts are involved in getting FNF web content delivered to our visitors. We are doing this with entirely open source software and commodity hardware.

Keep an eye on the blog for an announcement of our FreedomDataCenter release very soon. It’s been under way for about a year now, and we are close to release!

Thanks for reading. :)

FNF Technical Operations update

Hey everyone. It’s been quite a while since I’ve blogged for FNF.  It’s been a busy few months in techops. What did we get accomplished in Q3?

  • On boarded some very senior leadership and direct contributor talent to help bring FNF technical operations and web operations work much further along.
  • Deployed into a 42U cabinent in Joes Data Center. In that cabinet we:
    • Deployed redundant, managed power distribution units. Cyclades. 10 ports each.
    • Deployed a second Cisco 2950 switch. Not yet setup with spanning tree, but that’s the intent.
    • Deployed redundant pfSense systems. Dell Poweredge 2650 systems with 4GB RAM, each with dual power supplies (one going to each of our redundant PDUs), single hard drive (with almost no writes, since syslog goes to our central syslog server), single WAN/LAN link). Using CARP for automatic fail over. In our testing, routing failed over seamlessly and INCREDIBLY fast. Very impressed with pfSense. Who needs expensive cisco ASA systems, when pfSense does the job quite nicely!
    • Deployed redundant storage systems, using FreeNAS. Utilizing Dell Optiplex 745 systems with 4GB RAM and single 1TB hard drive, each with single power supply,NIC.  We are currently replicating from STOR01 to STOR02 on a continuous basis, which provides us with business continuity in the event of STOR01 failing. STOR02 is primarily intended for very IOPS heavy work loads in the dev environment, but also serves nicely as a (very close to) real time data replica.
    • Deployed a secondary virtualization server , primarily for product development/testing, but also as a failover server in the event our main VM server goes down.
  • Rebuilt our disaster recovery environment to better mirror our new production environment.
  • Migrated off of cPanel to a fully open source system.  Dedicated virtual machines for a number of applications (Jabber,BigBlueButton,Canvas LMS,iRedmai,FreeIPA,MySQL), and a Debian LAMP (with APC) system for our various thefnf.org subdomains.
  • Migrated to thefnf.org across all web properites.
  • Deployed a massively tuned MySQL database infrastructure (with replication to a local server and to our DR site)
  • Greatly enhanced the performance of all web properties by deploying Varnish + APC + memcached + mysql caching + zfs caching.
  • Deployed ZenOSS as a monitoring solution
  • Deployed OSSEC for host based intrusion detection and file integrity monitoring
  • Engineered and deployed a multisite data replication scheme to ensure that all systems are have nearline backups and offline archives.
  • Purchased a 32 port console server (Cyclades AlterPath) for 100% out of band access to all systems.

In short we now have a fully redundant,  enterprise grade system to move forward with. We’ve been heads down for the past month or so (since the summit) working on getting this finished.

As always, we do everything in an open source/transparent manner. Feel free to track our progress.

We’ve greatly improved our documentation, and invite you to check it out.

Thanks for reading!

An update from the CTO

It’s been quite a while since I’ve posted an update. I’d like to take this opportunity to let folks know the massive progress that has been made on the technical side. I’ve been serving in a combined CIO/CTO role supporting deployment of the FNF enterprise infrastructure. It’s been quite a while, but I’m finally transitioning over to serving full time as CTO and heading up product development.

So what have we been up to on the technical side?

1) Deployment of a disaster recovery system in the southwestern United States. This will provide us with continued operation of FNF enterprise infrastructure and FreedomNOC. The server has been deployed, initial data copy has been completed. We are in the process of turning up the virtual machines and setting up continuous replication.

2) Completing deployment of a full suite of applications to support enterprise operations. You can see the full list of deployed capabilities here:


We are in the process of having everything use LDAP for authentication/authorization. That is the last major project on the CIO side.

3) Additional folks are contributing in the areas of technical operations and product development. This is helping things move forward at a very quick pace.

4) FreedomLab is in the process of being deployed. The Austin lab (serving a continuous integration role) should be online very soon. We are waiting on our data center to work out a contract for an expanded service footprint which will allow us to deploy our WAN networking lab.

5) Lab network build out in KC. This will include radios, back haul, FreedomLink/FreedomTower. This will be part of the expanded service footprint that we are attempting to negotiate with the data center.

Ideally we will be doing an initial release of the full FreedomStack on July 4th.  June is going to be a very busy month!

FNF January Newsletter

It’s been an incredibly busy, challenging and rewarding year for the Free Network Foundation. I wanted to take some time and publish our first monthly newsletter  and show how the FNF went from a rough idea of myself and Isaac to a non profit start up company.

So what did we do in 2011? The following is in rough chronological order:

  • Isaac and Charles officially launched the FNF and had a week long summit to work through many of the details, come up with network plans, research hardware, figure out levels of the network etc.
  • Isaac traveled the eastern seaboard and abroad to raise awareness and funding for FNF.
  • Designed, built, deployed, tested, tweaked, revised three radio towers directly (two for OWS (including one taken during the OWS raid), one for OccupyAustin) and provided engineering, design and implementation support for two more (Los Angeles and San Antonio). That’s a total of 5 towers. 4 of which are in operation.
  • Awarded 10k innovation award at ContactCON at the end of October and a good amount of funding throughout the year
  • Received quite a bit of publicity in leading online publications and developed excellent relationships with online and print media
  • Deployed a beta of our Network Operations Center with full production release scheduled for end of January 2012
  • Deployed a beta of our Development and Test lab with full production release scheduled for end of January 2012
  • Assembled our executive leadership team (CEO/CTO/CIO/CFO/COO)
  • Put together a list of candidates for a board of directors and an advisory board.
  • Held a year end summit and review. Figured out a lot of logistical issues and also planned out re launch of website and did a lot of design work on FreedomNode. We also took steps to eliminate single points of failure on the executive leadership team. We now have technical/legal/operational/financial redundancy.
  • Published our roadmap to a 1.0 release of the FreeNetwork stack (our 2012 plan)


So what is ahead in 2012?

  • Our focus in January will be on finishing up the last pieces of scaffolding work for mass participation. Things like the NOC/Lab being completed and opened for use and our corporate structure to be fully operational. This includes things such as financial transparency, being able to accept non profit donations, legal things related to being a company etc.
  • Our focus in February will be on deploying our single sign on, one time password, authentication platform and getting folks to deploy a VM image which will replicate the one time password system. It will be a globally distributed, peer to peer architecture, highly available, democratized identity ownership platform. This will be our first release to end users. The authentication system is the first piece of the network. We also need to publish the tower plans and support an emerging community of builders and early adopters.

Beyond that we will be proceeding against our roadmap and building lots of cool stuff!