Hi, My father Mike, a former Town Manager has been helping us better understand what kinds of information are available within the walls of small city hall. I asked him - "What kinds of information might be there and who might be responsible for it?" Here is his list and notes.
List of things in municipal filing cabinets
Much communication is done by email, should get the folders for all these folks also depending on the size of the community, many of these specific categories of information will located in unexpected places. An example is purchasing information in the Town Manager’s area of responsibility. Knowing where these documents are located is important if the organization does not post up all docs as standard practice. In smaller towns the payroll and accounts payable info is often with Town Clerk employees (EE).
Getting copies of all the attachments or documents referenced in agendas will provide much of the information listed in the filing cabinets of the people listed below.
Important documents would be the personnel manual and labor agreements, these might be located anywhere as they would have to be approved by the legislative body, such as attachments or referred documents in agendas or minutes. These documents will answer many questions citizens have about how much people get paid and limitations on discipline when necessary. These might be in the custody of the Town Clerk and or Town Manager.
I created a Gist for the list below: https://gist.github.com/driki/5354198 if you have other ideas for where information and documents might be within our smaller towns please comment on the Gist. If it proves useful I'll find a more permanent home for it.
Meeting minutes with attachments Agenda Items with attachments
Town/City Manager and their secretary or assistants
Citizen complaints about almost anything someone has an issue with. Studies and surveys about services the community provides. Newsletters and other mass citizen communications Monthly Department Head reports Requests for use of public facilities, example is Seal Harbor LIbrary using the green to hold their annual book sale and picnic. Draft and in process budget Tea, coffee and snacks for working late
Town Clerk and assistants
Voter registrations, political party affiliations, voter lists. Census reports Demographic and economic data from the state and federal government Motor vehicle reports to the state. Birth and death records, many help with genealogy research. Annual Town Reports contain reports from all town departments and all affiliated groups such as the garden club and any community groups supported with taxpayer dollars.
Treasurer (Finance) and assistants
Accounts payable and payroll periodic payments, also part of agendas if payments must be approved by the elected officials and or the legislative body. Current budget status reports, reflecting payments above. Debt reports, payments for bonds and equipments leases. Lease approvals will also be part of agendas and minutes. Audit reports
Planning and Code Enforcement
Comprehensive Plans Building, plumbing and electrical codes Citizen complaints about neighbors, and action taken to respond to those complaints. Enforcement actions for code violations
List of properties, both taxable and non-taxable Reports to and from the state on quality of assessments. (most jurisdictions require a certain equity of assessment for properties based on quality, age and size of structures) Sales reports
Capital improvement plans (CIP), may also include Solid Waste, Water and Sewer in smaller communities. Equipment lists Road improvement and paving schedules with cost estimates Service Requests Contracts
Disposal and recycling data
CIP Quality reports to state or federal government Participation in clean water initiatives
Quality reports CIP
Purchasing, smaller communities likely part of Town Manager responsibilities
RFPs Cooperative purchasing agreements, (agenda item)
Fire inspections Equipment and readiness reports
Arrests Incidents Traffic Reports Concealed weapon permits Dispatch reports and 911 recordings FBI and state law enforcement reports
Organization Chart Personnel Policy Job Descriptions Labor agreements Disciplinary actions, this is a unique area, in Maine the only public document is the final action and not all details leading to the disciplinary action are public. Job postings and advertising
Acceptable use policies Website hosting agreements Website traffic reports IT inventories GIS mapping for Planning
Jason and I have been blown away.
We want to thank you.
Jason and I submitted our Knight Foundation News Challenge for Open Government application last week. We knew it was a good idea to ask our friends and former co-workers to help us with our submission. Our application is stronger and will stand out because of the help we received reviewing our early notes, hearing our story, editing our words and creating visual designs. If you need design, writing or just freaking smart people for a project please reach out to:
Khoi Uong - Pitch deck visual design
Adam Olenn - Editing, storytelling
Gina Mosca - Initial appeal editing
Kelly Gray - Appeal editing
Rich Jones - Bullshit detection
MJ Davis - Asking awesome questions & editing
We’re still not asking for money.
There will be over 1,000 applications, so we need to get Knight to pay attention to ours. The submission period closes in 9 days and we need to raise the visibility of our application before and during the review process.
You can help.
There are two simple ways you can help us at this moment:
Low effort (1 minute) - Tweet to @knightfdn and let them know you support what we are doing. A suggested Tweet might be: I think #opengov is important and want @NearbyFYI to exist. @knightfdn please make sure you look at: http://kng.ht/Zsmjyh #newschallenge
Low effort (10 minutes) - Read and review our online application and provide a public comment or question. Your public questions about our application are important and will help us refine our proposal.
We’ll start preparing our technical and financial presentations so that our tech and business friends can start peppering us with questions. We also know that it's important for the Knight selection committee to hear from our friends, so Matt will be reaching out to some of you for specific help.
Thanks, Matt & Jason
There is a great conversation taking place over at the Knight Foundation News Challenge website about open government data and methods for creating a sustainable, self-funding project and service. The primary debate is over paid vs. free access to the data that systems would collect. I pulled my long, really long comment out and wanted to share it here.
Warning, lots of I think statements coming up.
I think we can all agree that there are significant ongoing costs to supporting a system that aggregates, makes sense of and provides simple easy access to the millions of documents and billions/trillions of data points created by our local municipalities each year. Certainly the cost of cloud based tools have dramatically reduced infrastructure costs, but they are still very real. But, forget the infrastructure costs for a moment, a system like this will require some number of humans to set it up, maintain and improve it over time. It just won't run itself. So, who pays for this system and how? As we develop our business at NearbyFYI it'd be great to hear from others about what it might take to build and operate this system. Who values this data? We have our own assumptions but it'd be great to hear yours too.
I think we also all agree that citizens and municipalities should have access to this information. This is information that will benefit citizens and municipalities both. Eric's point and one that I agree with, is that very few municipalities currently have good analytical access to their own information and data. It's one of the reasons why I've had members of local governments reach out to us about what we are doing. So why hasn't this been done yet? I think the questions become: How is the municipal data made accessible to different entities and what restrictions, if any, are placed on the use of that data? How do individual municipalities make their data available? I have my opinions based on how I think a sustainable business can be built that lifts all data boats, including those of citizens, municipalities and corporations.
I think open standards and local legislation for tool vendors will provide for a level playing field. The legislation and standards will take time to be adopted though, and like most standards, probably will never get 100% buy-in. If multiple entities are able to access municipal information in a common standard format like Open311 or XBRL as Marc Joffe mentioned, then I'm not worried about a single entity having universal control over this data. Also, if we make it easy for each municipal data node to conform to a standard, then external parties will be able to aggregate it efficiently. Then the organization or company that can most efficiently collect the data and add the best additional value will attract the most users, citizens and customers. The question I think is relevant here is: Will municipalities adopt open data standards and force their vendors to use them?
I think over time that raw government data will become a commodity as becomes easier to access the data. If open data standards are adopted it will happen . At the moment though, in the absence of any standards and easy methods to access this local city and town data, there is still value and a need to collect it and provide simple aggregated access.
I think in the long run any company in this space will be working to add annotated data and value on top of this information and data. The raw data alone eventually will not be useful, much like none of us really interact with the raw GPS data, we use and pay for tools that add value for us. Don't get me wrong, I think if the municipalities all stood together and agreed that they needed to create the GPS equivalent of an open municipal data system, I'd back that. It'd save NearbyFYI a ton of work and I could skip over the boring parts to get to the good analytical stuff. I just don't see it happening. So rather than sit around and not have this type of information and data we decided to do something about it.
The current model, maybe the only model, that cities are using today is to pay a vendor like Socrata to stand up an "open data portal". Often those data portals cost upwards of $50,000 to setup and have ongoing annual fees. I'd also question the value of the data being provided, check out Data Boston not exactly a great set of data to work with. We're already paying to access this data, it's just hidden in our taxes and not evenly distributed.
Regarding the common good, if our company is selected by Knight for a grant, one of our first items will be to convert from our Massachusetts based C Corp into one of the newer Benefit Corporation entities that are starting like Etsy. While becoming a benefit corporation might not be exactly what you or others might want, I think it's a great solution to creating flexible corporations that want to address social improvements while still competing in the free market. I'd be happy to bake into our charter, mission, covenants and legal documents something along the lines that you are proposing. I want this to happen and think that having access to this information is extremely important.
Thanks, Matt MacDonald
At NearbyFYI we are flipping the existing open government data portal model and how citizens and companies get access to municipal information and data. We intend to provide municipalities with high quality, commercially supported, standards based open source software that replaces exiting proprietary vendor solutions.
Along with providing high quality tools with APIs that use or help develop standards, we're using existing public data collection methods to extract high quality information from municipalities. Combining this approach with ETL and Data Warehousing within a municipality means that we can start reducing the $54B that municipalities spend on IT, while dramatically improving access to the information for everyone.
There is valuable information in municipalities. There is value for citizens, corporations and also for non-profit/NGOs. Providing aggregated access to this information across municipalities increases the value that can be derived from all parties, including for the cities and towns. We've been doing this in Vermont, working with Vermont Public Radio to help them analyze meeting minutes, agendas and other municipal documents for every town in Vermont with a presence on the internet. You can see the output of that on their Public Post website. Vermont Public Radio pays us a small fee to access this aggregated information, our service replaces what was a very manual process. Jonathan Butler is our primary contant and awesome partner there, I'm sure he'd agree that it is well worth it for them to have this level of simple access.
There is often a debate about a free vs. commercial approach to providing access to this information, but after having worked on NearbyFYI for the past 8-12 months with my business partner, there needs to be a financial model for supporting this effort beyond non-profit grants and municipal funding. There are potentially many financial models to make this work. Our approach is to provide unlimited access to this information for any single municipality, for anyone, regardless if they are a citizen, commercial or non-profit, but if the user has a need for searching or accessing information across a large number of municipalities and deriving revenue or a profit, then fees should apply.
I think the real value is going to be in which company or organization can apply the best analysis of our municipal data, who is going to create the predictive analytics tools, the trend analysis, topic clustering and who will provide the best search interfaces. Raw municipal data should be freely and openly available, but anyone who adds additional value to that data, be it as simple as extracting text from PDFs to performing sentiment analysis on the documents and data or developing a domain specific Topic Model should be compensated for that work. It's likely that the information derived from those raw sources probably should be copyright to the organization that took the time to do the work. Copyright on this data doesn't remove access to citizens, it just means that it might have certain restrictions for use. Jason and I have families to feed, I have four mouths, if there is a way to feed them and keep provide amazing access to these documents that is entirely free to the public, citizens and companies both, It'd be great to know how to do that.
I think there is enormous opportunity to unlock the vast data that our cities and towns have within their control. Making this information visible, useful and accessible is something that must have an ongoing sustainable model. I don't think grant funding alone and depending on the community makes that happen. It might be useful to require that companies aggregating this information are are Benefit Corporations or Public Interest companies, it also could be baked into a covenant or the legal structure of the entity, that in the event of the company being shuttered, that the data collected, tools and software must be released back out to the public. That might prevent another EveryBlock from happening.
If you've reached the bottom of this post, it's likely that we'd have an awesome conversation. I can be reached at email@example.com or firstname.lastname@example.org.
Jason and I have amazing friends and former colleagues. You.
We need your help.
As many of you know, I've spent the better part of two years working in my spare time to disrupt and improve how our cities and towns function. My best friend Jason Howes has devoted the past 10 months working with me on NearbyFYI. Kim and my kids have been insanely patient and supportive, you have too. This is a moment for NearbyFYI to rise up and do something amazing.
We’re not asking for money.
We’re building an internet-based data analytics platform for municipalities that we know will help city and town employees run their communities more efficiently. Most of this information exists in the dark corners of the internet where PDFs and Word documents hide. We know that accessing this information is difficult; even Google can’t help you search it, and access won’t get any easier unless we do something about it.
Jason and I are applying to the Knight Foundation News Challenge for Open Government. The Knight Foundation typically gives out $300,000 or so in grants for projects of our size. We know we can be a contender. We’ve built a proven prototype, but we need this grant money to start the hard work -- scaling up to cover every municipality in the United States and improving our existing document analysis tools.
This is our moment.
You can help us craft an amazing story in our grant application, something that stands apart from everything else. There are bad grant applications, good ones and then there are the truly great ones. We're asking you, our gifted group of friends to help us make our application exceptional. Our network of friends is incredibly talented -- artists, musicians, writers, designers, filmmakers, engineers, storytellers and pranksters.
You can help.
- Storytellers & Smart People - You could help by talking with us for 15-30 minutes to shape our raw materials into a cohesive story outline
- Copy Editors & Writers - You could spend 15-30 minutes reading and editing our application in a Google doc
- Visual Designers - You could help by spending 2-4 hours sprucing up a 10-15 slide deck created by Matt
- Technical & Financial Peeps - You could help by spending 15-30 minutes poking holes in our technology and financial assumptions
- Anyone - You could help by taking 5 minutes to publicly back our application when it’s online
- Filmmakers - You could help by editing a script for a 2 minute interview style overview video
- Audio Producers - You could help by properly miking and mixing a 2 minute interview
- The application process starts on February 19th
- I expect to have a rough text draft in place by Monday March 4th
- The application must be submitted by March 18th
If you’ve gotten this far thanks. If you’d like to help or know anyone that would, please send an email to both me and Jason so that we can follow up: email@example.com and firstname.lastname@example.org We are planning to ramp up our proposal effort starting on the 28th.
Thanks, Matt & Jason
I saw this post/idea over at the Knight News Challenge - Open Government ideas site: https://www.newschallenge.org/open/open-government/inspiration/data-liberators-your-city-needs-you
"Data liberators: your city needs you Cities don't always understand the difference between a data catalog and open government. We need open software to light the way."
I agree with this 100%
Scraping 200,000+ documents from 100's of municipalities in Vermont has proven to me that open data portals aren't really what municipalities and citizens need. What is needed are much better municipal tools for concepts like https://github.com/driki/muni-meeting or http://www.budgetvision.com which are prototypes and proof of concept ideas that I've tried out over the past year. While finally building out NearbyFYI: http://said.nearbyfyi.com
I think this is a problem that needs to be addressed with two parallel paths, improved tools, and tools that do a better job of collecting the information already in these dark corners of the internet where PDFs, Excel files and PowerPoints hide.
Without drastically improving the internal workflow tools that municipal employees use, data portals are going to be less useful to citizens. I'd rather see the municipal legislative process take place online and the output happens to be better data than to have that data show up in a 'data portal' from the commercial companies that we're paying for.
Open Data portals, apps and engagement are sexy fun and often really pretty and flashy, the hard dirty work, the real hard work is going to be providing useful tools that improve the workflow of municipal employees and where the by product is improved citizen access and increased possibility of participation. Their current tools suck now.
My father, a former city manager said. "Citizens only care about four things:
1) Is my trash being picked up? 2) Does my shit go down the toilet when I flush? 3) Do I have clean water to drink? 4) How much are my taxes going to go up?
Shiny civic apps are great, the've done a good job of raising awareness about what this stuff means, the possibilities and are effective marketing tools, it's now time to focus our efforts on streamlining the municipal government process so that my service goes up and my taxes go down.
Thanks, Matt MacDonald.
Open Docket is a meeting management tool specifically designed for Municipalities and how they run public meetings. I've been working with and talking to with Sean Roche from Newton, MA about this problem space and how developers can help improve the local legislative process.
What problem is being solved?
Local government is too sexy. It gets people excited. We can't get enough of it.
OK, it isn't really any of those things, but it could be more efficient, simpler to understand and easier to participate. Much of the work that takes place in local government is done in Select Boards, Town Councils, School Boards and Sub-Committees like Finance and Transportation. Most people could care less what goes on in these meetings until their local property tax bill comes. Most cities and towns live in the stone age when it comes to the processes for running their meetings. It's like the Internet never even existed. On the left is a Word document created, saved, printed, scanned as an image and then turned into a PDF to be posted online. This is very common in our cities and towns and it has to stop. Having good, quality, structured data about the policies, ordinances and legislation is important as many state and federal laws have their roots in local legislation.
At NearbyFYI we've been working on a set of tools to collect meeting minutes, agendas and reports from hundreds of cities and towns in Vermont. We have over 150,000 documents now. We're doing the best that we can to extract meaningful, structured data from the blobs of PDFs, Word documents and the most poorly formatted HTML you've ever seen. We're finding useful, interesting bits of data in this local legislative soup, data that we think others are interested in paying for. Vermont Public Radio with their Public Post tool is using the information we're finding to write stories that have been picked up by NPR and the Associated Press.
We're never going to win the battle though. The upstream source is so polluted. We need to clean things up. Open Docket is an online tool specifically designed for Municipalities and how they run meetings.
Open Docket benefits for cities and towns:
- Reduce meeting taker and organizer time
- Real time publishing of notes - zero publish time
- eDelivery of Meeting Packets (Police Officers usually hand these out manually)
- No need to convert Word docs and flatbed scanner documents to PDF
- View voting histories, profiles
- Record meeting audio via iPhone and Android apps
- Meeting topic trends
- Searchable meetings
- Low-cost or FREE tool
- Open source, open APIs for data
- Useful, structured data and information for analysis
There is a closed source, commercial vendor in this space called Granicus. They build decent tools, they have an API (limited non-public access) but they create an expensive, closed and complicated tool. It is a tool designed for larger cities and towns. Towns with closed circuit camera systems and $40,000,000+ budgets. Most towns in the United States have fewer than 50,000 residents. These are the towns where a Selectboard meeting might take place in a library or in a kitchen of a member. These smaller towns pass important laws and ordinances that are rarely noticed in our busy lives. Democracy is happening in public view, but we just don't see it.
If you have gotten this far it's likely that you'd be interested in talking with us about we're hoping to do. We'd love to collaborate with others on this.
"My Administration is committed to creating an unprecedented level of openness in Government. We will work together to ensure the public trust and establish a system of transparency, public participation, and collaboration. Openness will strengthen our democracy and promote efficiency and effectiveness in Government." -- BARACK OBAMA http://www.whitehouse.gov/thepressoffice/TransparencyandOpenGovernment
App Vision, features and goals
Open Docket SHOULD allow two deployment modes, single user and multi-tenant. Most of the smaller communities in the United States are not likely to have the ability and resources to setup and maintain their own Open Docket instances, but those that can should have the ability.
Access to data within Open Docket MUST be accessible via APIs and bulk exporting. A production Open Docket instance must not turn public, programmatic access off.
Standalone Open Docket instances MUST be discoverable by other Open Docket instances.
Open Docket takes an opinionated approach to how public local government meetings are run. Each city and town has a unique process but there is significant overlap to them. Open Docket is designed to solve 90% of the overlapping concerns, it is not intended to be tailored for each specific workflow.
Open Docket is designed for the 30,000 cities and towns in the United States with fewer than 50,000 residents. While it SHOULD and likely will support the facilitation and meeting process of larger cities and towns, it isn't intended to address larger, complicated legislative processes.
Meetings can often take place in low/no network connectivity locations, especially in more rural towns without public WiFi. The Open Docket tools should take this into account and when possible use local storage so they can be used without network connectivity.
Ideally an API specification and data exchange format for for Open Meetings develops similiar to what Open311 has done for call center related activities.
Proposed Models & Roles
- Municipalities like Watertown, MA or Cambridge, MA have many Organizations
- Organizations like Town Council, School Board, or the Finance Committee have many Members
- A Member can belong to many Organizations, like Matt is the Chair of the Finance Committee and the Vice-Chair of the Transportation Committee
- A Guest is someone who attends a Meeting but isn't a Member of the Organization running the Meeting
- Each Municipality has a list of Administrators that can create, edit and manage Organizations, Meetings & People
- Members, Guests and Administrators are all required to have First & Last Names
- Members & Administrators must have contact information including email, phone and street address
- Guests should have an address and optionally contact information
- Organizations have many meetings
- Organizations have Regular Meetings on a defined schedule
- Each Meeting often has an Information Packet that needs to be delivered to Members
- A Meeting follows a Meeting Template or Meeting Script
- Each Organization might have a different Meeting Template
- Each Meeting has many Agenda Items
- Each Agenda Item can have many Attachments like PDFs, Word docs, etc.
- Each Meeting has a Scribe often referred to as a Secretary, Note Taker or Minute Taker
- Motions are made on some Agenda Items and the Votes of Members are recorded
- Each Meeting has Minutes which are recorded by the Scribe and are often required to be posted publicly
Municipal Meeting Workflow & Process
For each Organization Meeting that takes place an Agenda is created, usually by the Scribe (referred to as the Secretary in most cities). The Scribe constructs an Agenda that looks very similar to previous meetings and often they maintain a Template of the Meeting in Microsoft Word so they can copy it and quickly get a new one created. The Scribe enters a number of new Agenda Items into the template to create a DRAFT Agenda. Agendas across Municipalities follow a typical structure:
- Call to Order - Bob opened the meeting at 6:32pm.
- Roll Call - Matt: Present, Jason, Present, Chris: Absent
- Executive Session
- Pledge of Allegiance
- Amending of and/or Adoption of prior Minutes - Jason moved to accept the October minutes
- Public Comments/Forum - An opportunity for those in attendance to speak about non-Agenda Items
- Agenda Items unique to this Meeting - Motions can be made and Votes are tallied
- Public Comments/Forum - An opportunity for those in attendance to speak about non-Agenda Items
- Closing Remarks
- Meeting Adjourns
The Scribe is responsible for Organizing the Meeting, posting Public Notices to Newspapers, websites and other Media outlets. When the Agenda has been set and supporting documents are ready the Scribe will send out the Meeting Packet to Members of the Organization responsible for the Meeting. Often this is only done for Selectboard and Town Council Meetings as the smaller sub-committees don't have the resources to prepare for Meetings like this. Typically the Scribe hands the Meeting Packet to a Police Officer who then will physically drive to Members homes and hand deliver the Meeting Packet.
Organization Members are responsible for reading and preparing for the coming Meeting, though often many Town Councilors or Selectboard members will often show up to a Meeting ill-prepared and not having had time to read the materials until just moments before the Meeting. There are Town Councilors that fight e-Delivery of Meeting Packets and request that manual, paper deliver still takes place.
Municipal Government will likely take considerable time to transition to digital only workflows so any Open Docket documents MUST provide downloadable, printable formats.
The Actual Meeting
Meetings typically take place in the evening, usually starting between 6:00-7:00pm. A Meeting is officially called to order by the Chair-Person for the Meeting and the official start time is recorded for the record. Roll Call is usually initialed by the Scribe and is when Members declare their attendance to the Meeting. A verbal record of Present or Absent is recorded by the Scribe. A Scribe typically takes notes either with paper and pencil or with a computer using Microsoft Word. In communities of 30,000+ there is often a cable access feed provided of the larger meetings. Smaller towns and sub-committee meetings typically have no audio or video recording available.
Typically a Meeting is called to order & roll call taken only to have the Meeting move into Executive Session, where the contents of the meetings are private within the constraints of a states Open Meeting Laws. After returning from Executive Session (typically 15-60 minutes) the meeting beings in front of the Public. The Pledge of Allegiance is often performed.
Their is usually an Agenda Item to review prior minutes which often results in no amendments. There are times when Amendments are requested by Members and the Scribe is responsible for noting them and adjusting prior minutes at a later date.
During the Meeting the Scribe takes notes for each Agenda Item which are later posted as Meeting Minutes. Typically there is a presentation of each Agenda Item, either by a Member or Guest who has been either requested to speak or who has requested to speak. Agena Items are typically set 3 days in advance of a Meeting so that proper public notice can be provided for those that wish to attend.
Usually the first Agenda Item after the Pledge of Allegiance has been performed is a Public Comment or Forum period. Each Guest in the audience is given the opportunity to speak about any topic for a maximum time period. Typically the time allotted is between 3-5 minutes. Most Municipalities require that a Guest speaking during Public Forum state their name and street address. This is usually recorded by the Scribe but there are often typos and missed information because of challenges hearing new names and street addresses.
Often a Scribe will refer to Members and Regular Guests by their initials when taking notes. For example, Matt MacDonald would often be referenced as (MM) throughout the document.
After a presentation of an Agenda Item from a Guest, the Chair of the Meeting will typically open the floor for discussion among fellow Members. Members are given the opportunity to comment and ask questions of the Guest presenting. After discussion, the Chair asks if there are any motions on this item to be made. A Member of the Organization makes a motion and waits for it to be seconded by another Member. After the motion is seconded a vote takes place. If the motion fails to seconded no vote takes place. When voting, the Scribe verbally calls out to each Member of the Organization and asks for their vote of Yay, Nay or Abstain. The Scribe records the vote for each Member and the Chair of the Meeting announces if the motion passes or the motion fails.
This process is repeated until there are no more Agenda Items, the next public Meeting is announced and the meeting is adjourned with the official time recorded.
The Scribe will often sign or write their name.
Post Meeting Process
Meetings are typically created in Microsoft tools like Word or written by hand on paper. Attachments for supporting materials are typically PDFs, PowerPoint or Excel documents. The Scribe is often responsible for converting their Microsoft Word document into a PDF that then needs to be posted to the web. Often the Scribe doesn't have the technical access or ability to post to the municipal website. More meeting minutes, agendas and documents are being posted to the web as HTML documents, while the text is easier to access the raw meeting text is usually inaccessible as the templated, wrapper navigation and links are hard to separate from the minute text.
Often, larger cities using a CMS are able to have index pages dynamically created, but often the index of prior meetings are managed by hand in long, large documents that are error prone.
Meeting Minutes are in a DRAFT state until they are approved by the Members at the following Meeting.
Other Challenges for Municipalities
- Most Meeting Minutes are not searchable, few cities or towns have useful search systems that index the content within the PDFs and Word documents currently published.
- Most Municipalities have now database of votes from Town Councilors and Selectboard members. Citizens are unable to track how their local elected officials have voted. There is no record.
- Providing Meeting Packets is a manual process and there are time savings to be had.
- Each Scribe takes minutes in differing formats, using different date formats and templates.
- No web accessible historical record. Often cities bury this information in dark corners of the web.
I wanted to give you an update on NearbyFYI this week.
Over the weekend I released some updated code to improve our auto-classification of the documents that we are finding. A big push though was around outreach to the developer community to rally support and assistance with building a tool that allows municipalities to manage their meetings.
A memory consumption issue that we identified about two weeks ago with http://said.nearbyfyi.com has been resolved. We're also in the process of swapping out Open Calais for the free/open source Stanford Named Entity Extraction tools (NER).
I'm likely to have a quick conversation with Ben Berkowitz, the CEO of http://www.seeclickfix.com - a tool to: Report neighborhood issues and see them get fixed. They have picked up quite a bit of press in the past 3 years and are one of the most mentioned OpenGov/Gov2.0 success stories. Ben reached out to me on Twitter to talk: https://twitter.com/neocMatt/status/290055227195944960
On a whim I tweeted Marc Cuban (https://twitter.com/neocMatt/status/290187989915680768) and Robert Herjavec (https://twitter.com/neocMatt/status/290190129971204096) no response ;)
On Friday night I wrote an email to the Boston Ruby mailing list explaining the challenges that I think can be solved: Local Government is Sexy https://groups.google.com/forum/?fromgroups=#!topic/boston-rubygroup/_oyRzpypOoA The goal for this email was to see if there are other developers in the area that are interested in creating an open solution to improve the local government meeting process. I received 3 public and 2 private responses. One from a developer in Watertown who is a Rails and iOS developer!: https://twitter.com/slothbear
The response from the developer community has been very positive. I'm thinking of how to pull them in closer to actually start converting interest into development work.
Knight is going to formally announce the next News Challenge - Open Government soon and we're still planning on submitting an application.
I wrote an overview of the Muni Meeting tool and started to flesh out the need, potential solution and provide information about Models and actions performed. Also a start of wireframes and an issues list so that others can start to see. I created a new GitHub project that describes things here: https://github.com/driki/muni-meeting/blob/master/README.md
At NearbyFYI we review online information and documents from hundreds of city and town websites. Our (CityCrawler)[http://www.nearbyfyi.com] service has found and extracted text from over 100,000 documents for the 170+ Vermont cities and towns that we track. We're adding new documents and municipal websites all the time and we wanted to share a few tips that make it easier for citizens to find your meeting minutes, permit forms and documents online. The information below is written for a non-technical audience but some of the changes might require assistance from your webmaster, IT department or website vendor.
Create a unique web page for each meeting
Each city or town meeting that occurs should have it's own unique webpage for the agenda and meeting minutes. We often see cities and towns creating a single very large webpage that contains an entire year of meeting minutes. This may be convenient for the person posting the meeting minutes online but presents a number of challenges for the citizen who is trying to find a specific meeting agenda or the minutes from that meeting.
Here is an example of meeting minutes that are in a single page that requires the citizen to scroll and scroll to find what they are looking for: http://www.shrewsburyvt.org/sbminsarchive.php This long archived page structure also presents challenges to web crawlers and tools that look to create structured information from the text. Proctor, VT provides a good example for what we look for in a unique meeting minutes document. We like that this document can answer the following questions:
1. Which town created the document? (Proctor) 2. What type of document is this? (Meeting Minutes) 3. Which legislative body is responsible for the document (Selectboard) 4. When was the meeting? (November 27, 2012 - it's better to use a full date format like this) 5. Which board members attended the meeting? (Eric, Lloyd, Vincent, Bruce, William)
The only thing that could improve the access to this document is if it was saved as a plain text file rather than a PDF file. Creating a single web page or document for each meeting means that citizens don't have to scan very large documents to find what they are looking for.
Save PDFs as text not images
After running our CityCrawler for several months it's clear that cities and towns love the PDF file format to share information online. While PDF files can be a quick way to post information online, cities are often publishing documents that are scanned images which is no better than taking a photo of the document and posting it on Instagram.
The challenge here is that search engines must use OCR Optical Character Recognition software to try and extract the text from the image. Anything that makes it harder for search engines to index your documents means that fewer citizens are going to find your published information. At NearbyFYI we often see documents that could easily be saved as a text PDF but are scanned as images. Here is how this document could easily be converted to a format that search engines can use.
Steps to convert your PDF to text
1. Scan the meeting minutes document and save it as a PDF. 2. Open the scanned document in Adobe Acrobat. 3. Select "File> Export> Text> Text Plain." 4. Name the document and click "Save." 5. Open the saved file and review for conversion errors. 6. Save the corrected document and post to your website.
Allow web crawlers in Robots.txt
Most information online is found via search tools like Google or Bing. Google uses what is called a web crawler or web page indexer to review your website documents so that they can add your content to their search index. This is a good thing, you want the search companies to find your content as it's likely the way most citizens are going to look for information about your community.
Robots.txt files contain a simple set of rules that web crawlers follow. Some websites are setup to allow crawlers others aren't. This is an example of a robots.txt file from a city who's online information won't be found with a Google Search:
User-agent: * Disallow: /
What this means is that when a citizen searches Google for "how to get a zoning permit in pownal, vt " they won't find this page: http://www.pownalvt.org/planning-commission/town-plan/zoning-rules/zoning-permit-application/. Ensuring that web crawlers can access the documents you post on your website is likely the simplest thing that you can do to improve citizen access.
Jason and I have been working at NearbyFYI on the CityCrawler service so that we can locate and extract structured information from the public documents and data that our cities and town and create. We provide an Application Programming Interface (API) as well as a website that makes it easier to locate documents from hundreds of cities. Our experience in reviewing thousands of local government documents and hundreds of city websites means that we can help you improve access for your citizens.
If you are interested in learning more about how your city can improve access to meeting minutes and other public documents we'd love to hear from you at email@example.com.
Matt & Jason
We're excited to announce that we now have public documents for every city and town in Vermont with a website. This means that developers and citizen hackers interested in building municipal or citizen focused applications or tools have access to structured public data from thousands of PDFs, Word documents and text files.
You could use the data aggregated by NearbyFYI to build a state wide bid and RFP notification service: http://said.nearbyfyi.com/api/v1/documents?state=vt&classification=bid or maybe create a tool that reviews meeting minutes that have happened since October. We're looking forward to how others might use the data that we're providing but I wanted to highlight one new use.
Vermont Public Radio :: Public Post
Vermont Public Radio has been working on the amazing Public Post project that helps raise the visibility of legislation and activity at the local government level. VPR is using the public data aggregated by NearbyFYI to help augment statewide reporting and identify potential stories for their journalists. The data provided by the NearbyFYI API allows VPR to cover more cities and towns and gives them access to thousands of public documents. You can see the output of their coverage at the awesome Public Post.
What's coming next?
- Adding search capabilities to the API
- Reducing noise and increasing signal (better quality docs)
- Improving classifications
- Adding documents and data from other municipal sources, school boards, public safety
If you are interested in learning more about how you can use the NearbyFYI API to create value for your customers or employees we'd love to talk to you. firstname.lastname@example.org.