We compared the effects of two diets on glycated hemoglobin (HbA1c) and other health-related outcomes in overweight or obese adults with type 2 diabetes or prediabetes (HbA1c>6%). We randomized participants to either a medium carbohydrate, low fat, calorie-restricted, carbohydrate counting diet (MCCR) consistent with guidelines from the American Diabetes Association (n = 18) or a very low carbohydrate, high fat, non calorie-restricted diet whose goal was to induce nutritional ketosis (LCK, n = 16). We excluded participants receiving insulin; 74% were taking oral diabetes medications. Groups met for 13 sessions over 3 months and were taught diet information and psychological skills to promote behavior change and maintenance. At 3 months, mean HbA1c level was unchanged from baseline in the MCCR diet group, while it decreased 0.6% in the LCK group; there was a significant between group difference in HbA1c change favoring the LCK group (-0.6%, 95% CI, -1.1% to -0.03%, p = 0.04). Forty-four percent of the LCK group discontinued one or more diabetes medications, compared to 11% of the MCCR group (p = 0.03); 31% discontinued sulfonylureas in the LCK group, compared to 5% in the MCCR group (p = 0.05). The LCK group lost 5.5 kg vs. 2.6 kg lost in MCCR group (p = 0.09). Our results suggest that a very low carbohydrate diet coupled with skills to promote behavior change may improve glycemic control in type 2 diabetes while allowing decreases in diabetes medications. This clinical trial was registered with ClinicalTrials.gov, number NCT01713764.