Want to use toll roads on the East Coast, but hate having to carry change? EZ Pass offers cashless, open road tolling across 17 states. The catch? States using the system might also be watching you. Although the system is intended to be used solely to collect tolls, several states, including New York and Virginia, are using the system to track traffic patterns and drivers–much to the alarm of privacy advocates.

It is virtually impossible to separate tolling information from the knowledge of where a driver has gone. Traditionally, drivers took a ticket when entering a toll road and paid it when they exited. EZ Pass and other cashless tolling systems simplified this process by replacing toll booths and tickets with digital transponders than can be pinged wirelessly.

The tolling industry says that tracking concerns are overblown.

“Transponders are only being read at the tolling facility,” Bill Cramer, communications director for the International Bridge, Tunnel & Turnpike Association (IBTTA), tells InsideSources. “Tolling agencies keep that information very confidential. It is only kept a certain number of days and then erased. I would also point out that we are pretty much tracked wherever we go these days by our cell phones.”

However, this type of system still leaves a digital record, and drivers are increasingly concerned about who has access to this information. One blogger connected his EZ Pass reader to a device that lit up and mooed each time it detected signals at the same frequency as the EZ Pass readers. A quick trip through Midtown Manhattan showed that the device was being tripped repeatedly in areas far from any toll roads.

In 2014, the New York Civil Liberties Union (NYCLU) followed up on this stunt with FOIL requests about E-ZPass readers to the New York City Department of Transportation, New York State Department of Transportation, New York State Thruway Authority and NYPD. They found that EZ Pass readers were located in the city in places other than toll booths, including nearly every intersection in Midtown.

“Through FOIL responses, we learned that city and state transportation agencies have set up E-ZPass readers in various locations to conduct traffic studies,” the NYCLU wrote of its findings. The program generated hundreds of thousands of records each day. While the agencies claimed that information identifying the drivers of the vehicles was separated from location data before it was analyzed, the NYCLU remained concerned abut the implications of mass surveillance.

“A traffic study is a laudable project, but adopting technology that allows for mass collection of individual vehicle trip time data is concerning when there are no appropriate privacy protections,” they said. “The New York State Department of Transportation and the New York State Thruway Authority, for example, produced privacy policies that are vague and barebones.”

A bill introduced in the New York state legislature to prohibit the disclosure of toll records for purposes other than law enforcement died in committee this summer. The trouble seems to be that the transponders provide departments of transportation with valuable information about traffic patterns.

Nor is New York the only state using toll transponders for more than just collecting tolls. The toll system in Massachusetts contains a “hot list” feature capable of alerting law enforcement when certain transponders or license plates pass through the system. The feature is supposed to only be used for public safety emergencies, such as a missing child, but has raised concerns.

After all, Maryland, New York, Pennsylvania, West Virginia, and Rhode Island use the EZ Pass system to monitor driver speeds. In most of these states, tolling agencies lack the authority to issue speeding tickets. When a car is recorded as speeding, they may receive a warning, but not a ticket. Generally, repeated violators can have their EZ Passes revoked, but this rarely happens.

Privacy worries might be mitigated if drivers had alternate options to use toll roads. These are dwindling as more and more states remove toll booths for cashless systems, which they say are both cheaper and safer. New roads may never have had them to begin with. As legislators look to tolls to pay for repairs to aging roads and bridges, drivers may be forced to get used to the idea that their EZ Pass is not just traveling with them, it’s watching them.

Follow Erin on Twitter.