Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Unity Catalog] - Research update creation of folder paths in ADLS to support Unity Catalog #981

Closed
thesqlpro opened this issue Dec 19, 2024 · 3 comments

Comments

@thesqlpro
Copy link
Contributor

thesqlpro commented Dec 19, 2024

Potentially NOT NEEDED - if we can create the Date/Time dimensions programmatically (task #984 ) then we can potentially stop seeding this data and remove the shell scripts that are creating adls folder paths
Task for #765

@thesqlpro
Copy link
Contributor Author

added as DoD for task #984

@thesqlpro
Copy link
Contributor Author

Data seeding needs to be done for those date/time dimension files as they are being used in the SQL Dedicated Pool.

@thesqlpro
Copy link
Contributor Author

We can potentially resolve this by having the same/similar code used to create the Date/Time dimensions in Spark to be used to create those dimensions inside of the SQL Dedicated Pool. This would prevent the lengthy upload and need to even have these files. It also allows for more dynamic creation of date/time times.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant