Limited Results When trying to Access Delta Shared Tables from C#

Shawn_Eary
Contributor

This C# code only seems to return about 8 rows when it should return 100:

// Per Databricks AI, you must run the following command if you don't
// want to get an error about delta.enableDeltionVectors from C#:
// ALTER TABLE [myCatalog].setest.billboard_hot_100 SET TBLPROPERTIES (delta.enableDeletionVectors = false);
//
// This code was mostly written by Databricks AI but some other AI tools
// were used.
using System.Net.Http.Headers;

class Program
{
  private static readonly string bearerToken = "[myBearerToken]";
  private static readonly string endpoint = "https://[myRegion].azuredatabricks.net/api/2.0/delta-sharing/metastores/[myGUID]/shares/seary_billboard_100_test_share/schemas/setest/tables/billboard_hot_100/query";

  static async Task Main(string[] args)
  {
    using (HttpClient client = new HttpClient())
    {
      client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", bearerToken);
      client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

      var content = new StringContent("{\"responseFormat\": \"delta\"}", System.Text.Encoding.UTF8, "application/json");
      HttpResponseMessage response = await client.PostAsync(endpoint, content);

      if (response.IsSuccessStatusCode)
      {
        string responseData = await response.Content.ReadAsStringAsync();
        Console.WriteLine("Response Data:");
        Console.WriteLine(responseData);
        File.WriteAllText("c:\\Users\\seary\\Downloads\\billBoard100-deltaShareResults.json", responseData);
      }
      else
      {
        Console.WriteLine($"Error: {response.StatusCode}");
        string errorData = await response.Content.ReadAsStringAsync();
        Console.WriteLine("Error Data:");
        Console.WriteLine(errorData);
      }
    }
  }
}

Also, the returned JSON seems slightly corrupt to me. How can I get the above code to return all 100 rows from 

seary_billboard_100_test_share.setest.billboard_hot_100?