By Max Lewontin

Washington

President Obama’s proposed federal college-ratings system is set to be released in time for the 2015 academic year, but if the comments from administrators and researchers at a hearing on Friday are anything to go by, the plan appears to be far from complete.

The ratings system will examine colleges based on measures of access for low-income and first-generation students, on affordability, and on student outcomes. It remains controversial because the ratings could eventually be tied to colleges’ access to federal aid.

At the public-comment hearing—held by the federal Advisory Committee on Student Financial Assistance, which will release a report to Congress on the ratings system at the end of the month—participants repeatedly raised a number of concerns about the plan. Here are four key questions that came up most often:

Will the Ratings Turn Into Rankings?

There’s an important distinction between those two concepts. Several participants questioned whether federal ratings would simply become an alternative to commercial college rankings—either by assigning basic letter grades or by comparing a wide swath of two-year, four-year, public, and private institutions with one another.

“We will not rank schools, nor will we publish a list,” said Jamienne S. Studley, deputy under secretary of education. “We will find the best possible data to find institutions that promote affordability, access, and outcomes.”

But different constituents want different things out of the ratings.

Several participants, representing a variety of public and private colleges, said the ratings should focus on consumer-friendly information meant to help students and families choose an appropriate college. Others, from a variety of think tanks and policy groups, said the system should instead measure colleges in terms of “institutional accountability,” by looking more closely at how they use federal funds, which students enroll, and which ones graduate.

How Will ‘Failing’ Colleges Be Dealt With?

Potential sanctions under the ratings plan came up repeatedly. There’s considerable concern that many colleges—including those that do not graduate many students; those, such as tribal colleges, that enroll a particular type of student; and those with small numbers of low-income students, as measured by Pell Grant recipients—will be punished for failing to meet certain metrics.

Sanctions are a particular concern for community colleges that serve primarily as vehicles for students to transfer to four-year colleges. Those colleges tend to enroll more low-income and first-generation students—a goal of the ratings program—but their transfer rates bring down their graduation rates.

“It is ridiculous to give, for example, in the Boston area, Harvard College a pass considering their resources and endowment but sanction Roxbury Community College, which has struggled with its finances and leadershipin the past,” said David S. Baime, senior vice president for government relations and research at the American Association of Community Colleges.

Are the Data Flawed? (And Are We Using the Right Data?)

Existing rating systems, such as the Education Department’s College Scorecard, tag some colleges with inaccurate graduation rates because of “faulty data or human error,” according to Jee Hang Lee, vice president for public policy and external relations at the Association of Community College Trustees.

But even if the ratings plan cuts down on errors, there are questions about which particular types of data will be incorporated, as well as how colleges will be grouped together.

For example: Using data on graduates’ earnings as a criterion for measuring college success would hurt tribal colleges because many American Indians live far from major manufacturing centers, on reservations with high levels of unemployment, said Carrie L. Billy, president of the American Indian Higher Education Consortium.

Why the Rush?

Even though the data can be problematic, it’s the right time for the department to move forward with the ratings system, said Michelle Asha Cooper, president of the Institute for Higher Education Policy.

But Ms. Cooper said the department should rethink how it will promote the new ratings. “The Obama administration should not simply create another tool that simply resides on a website,” she said. “The administration could be much more proactive in terms of its outreach through social media.”

Not everyone is equally eager to strike while the iron is hot. Sara Goldrick-Rab, a professor of education-policy studies and sociology at the University of Wisconsin at Madison, questioned the department’s intention to meet a fall 2014 deadline. She cautioned that including community colleges in the ratings system, for example, would have unintended consequences such as making the sole college in an“education desert” undesirable to students.