Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Databricks Community
- Data Engineering
- DLT: Schema mismatch error
Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
DLT: Schema mismatch error
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
yesterday
Hi
I am encountering the following error when writing a DLT pipeline. Here is my workflow:
- Read a bronze delta table
- Check Data Quality Rules
- Write clean records to a silver table with defined schema. I use TRY_CAST for columns where there is mismatch between the inferred schema from bronze table to the target schema. Though coding intensive, this approach has worked.
I am seeing suddenly an error that I cannot resolve. I have repeatedly examined the schemas and cannot seem to find any difference. Can someone explain why there is mismatch.
com.databricks.pipelines.common.errors.DLTAnalysisException: Table 'jcjob_table' has a user-specified schema that is incompatible with the schema
inferred from its query.
Declared schema:
root
|-- JOB_COMP_CODE: string (nullable = true)
|-- JOB_CODE: string (nullable = true)
|-- JOB_CTRL_CODE: string (nullable = true)
|-- JOB_NAME: string (nullable = true)
|-- JOB_CUST_CODE: string (nullable = true)
|-- JOB_CONTRACT_CODE: string (nullable = true)
|-- JOB_WIP_DEPT_CODE: string (nullable = true)
|-- JOB_WIP_ACC_CODE: string (nullable = true)
|-- JOB_LBC_DEPT_CODE: string (nullable = true)
|-- JOB_LBC_ACC_CODE: string (nullable = true)
|-- JOB_LTC_DEPT_CODE: string (nullable = true)
|-- JOB_LTC_ACC_CODE: string (nullable = true)
|-- JOB_CC_DEPT_CODE: string (nullable = true)
|-- JOB_CC_ACC_CODE: string (nullable = true)
|-- JOB_BILL_DEPT_CODE: string (nullable = true)
|-- JOB_BILL_ACC_CODE: string (nullable = true)
|-- JOB_BILL_AMT: float (nullable = true)
|-- JOB_COST_METH_CODE: string (nullable = true)
|-- JOB_STATUS_CODE: string (nullable = true)
|-- JOB_ACCMETH_CODE: string (nullable = true)
|-- JOB_CERTIFY_CODE: string (nullable = true)
|-- JOB_SIZE_CODE: string (nullable = true)
|-- JOB_WM_CODE: string (nullable = true)
|-- JOB_ACTION_CODE: string (nullable = true)
|-- JOB_EST_START_DATE: date (nullable = true)
|-- JOB_ACT_START_DATE: date (nullable = true)
|-- JOB_EST_COMPL_DATE: date (nullable = true)
|-- JOB_ACT_COMPL_DATE: date (nullable = true)
|-- JOB_LST_ADDON_DATE: date (nullable = true)
|-- JOB_LST_REC_DATE: date (nullable = true)
|-- JOB_REC_AMT: float (nullable = true)
|-- JOB_HB_REC_AMT: float (nullable = true)
|-- JOB_REVREC_AMT: float (nullable = true)
|-- JOB_HB_AMT: float (nullable = true)
|-- JOB_PROFREC_AMT: float (nullable = true)
|-- JOB_LST_REC_PC: float (nullable = true)
|-- JOB_LOC_CODE: string (nullable = true)
|-- JOB_BUDG_UNIT: float (nullable = true)
|-- JOB_COMPL_UNIT: float (nullable = true)
|-- JOB_DISB_AMT: float (nullable = true)
|-- JOB_REVREC_LST_PC: float (nullable = true)
|-- JOB_REVREC_PC: float (nullable = true)
|-- JOB_CONTRACT_AMT: float (nullable = true)
|-- JOB_COST_FLAG: string (nullable = true)
|-- JOB_BILL_FLAG: string (nullable = true)
|-- JOB_SUB_FLAG: string (nullable = true)
|-- JOB_SEC_GROUP: integer (nullable = true)
|-- JOB_BUDGCST_SAME_LEVEL_FLAG: string (nullable = true)
|-- JOB_WBSV_CODE1: string (nullable = true)
|-- JOB_WBSV_CODE2: string (nullable = true)
|-- JOB_WBSV_CODE3: string (nullable = true)
|-- JOB_WBSV_CODE4: string (nullable = true)
|-- JOB_WBSV_CODE5: string (nullable = true)
|-- JOB_WBSV_CODE6: string (nullable = true)
|-- JOB_WBSV_CODE7: string (nullable = true)
|-- JOB_WBSV_CODE8: string (nullable = true)
|-- JOB_WBSV_CODE9: string (nullable = true)
|-- JOB_WBSV_CODE10: string (nullable = true)
|-- JOB_WBSV_CODE11: string (nullable = true)
|-- JOB_WBSV_CODE12: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG1: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG2: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG3: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG4: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG5: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG6: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG7: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG8: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG9: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG10: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG11: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG12: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG1: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG2: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG3: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG4: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG5: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG6: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG7: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG8: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG9: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG10: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG11: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG12: string (nullable = true)
|-- JOB_BID_FLAG: string (nullable = true)
|-- JOB_BID_STATUS_CODE: string (nullable = true)
|-- JOB_BID_CODE: string (nullable = true)
|-- JOB_MAKEUP_FLAG: string (nullable = true)
|-- JOB_PM_FLAG: string (nullable = true)
|-- JOB_BILL_METH_CODE: string (nullable = true)
|-- JOB_INV_FORMAT_CODE: string (nullable = true)
|-- JOB_BID_SUBMIT_DATE: date (nullable = true)
|-- JOB_PROPERTY_ID: integer (nullable = true)
|-- JOB_AREA_DISTRICT: string (nullable = true)
|-- JOB_PROVINCE_CODE: string (nullable = true)
|-- JOB_REVENUE_GEN_STATE: string (nullable = true)
|-- JOB_UNBILLED_REV_DEPT_CODE: string (nullable = true)
|-- JOB_UNBILLED_REV_ACC_CODE: string (nullable = true)
|-- JOB_HIER: string (nullable = true)
|-- JOB_CHG_SEQ_NUM: integer (nullable = true)
|-- JOB_BUDGR_SEQ_NUM: integer (nullable = true)
|-- JOB_AUTH_RQ_SEQ_NUM: integer (nullable = true)
|-- JOB_SI_SEQ_NUM: integer (nullable = true)
|-- JOB_REVREC_CURR_DATE: date (nullable = true)
|-- JOB_REVREC_LST_AMT: float (nullable = true)
|-- WORK_LOC: string (nullable = true)
|-- JOB_MUTLI_OVHD_PC_FLAG: string (nullable = true)
|-- JOB_WORK_LOC: string (nullable = true)
|-- JOB_POLICY_NO: string (nullable = true)
|-- JOB_PREVAILING_WAGE: string (nullable = true)
|-- JOB_PL_POLICY_NO: string (nullable = true)
|-- JOB_FULLY_PAID_INVS: integer (nullable = true)
|-- JOB_DAYS_OUTST_INV_PAID_TTL: integer (nullable = true)
|-- JOB_RATE_BY_JOB_FLAG: string (nullable = true)
|-- JOB_USE_PAY_BILL_RATE_FLAG: string (nullable = true)
|-- JOB_USE_EQP_BILL_RATE_FLAG: string (nullable = true)
|-- JOB_CUST_CONTACT_NAME: string (nullable = true)
|-- JOB_TERM_CODE: string (nullable = true)
|-- JOB_INVOICE_GROUP_CODE: string (nullable = true)
|-- JOB_BILLING_TYPE_CODE: string (nullable = true)
|-- JOB_INVOICE_FORMAT_CODE: string (nullable = true)
|-- JOB_CONSTRUCTION_VALUE: float (nullable = true)
|-- JOB_MAX_HOURLY_RATE: float (nullable = true)
|-- JOB_MAX_BILLING_AMT: float (nullable = true)
|-- JOB_MAX_BILLING_BUDGET_AMT: float (nullable = true)
|-- JOB_BILLING_RATE_TABLE_CODE: string (nullable = true)
|-- JOB_IB_ALLOW_FLAG: string (nullable = true)
|-- JOB_IB_FULL_TARIFF_FLAG: string (nullable = true)
|-- JOB_CONT_TYPE_CODE: string (nullable = true)
|-- JOB_LONG_CODE: string (nullable = true)
|-- JOB_RESERVE_REV_DEPT_CODE: string (nullable = true)
|-- JOB_RESERVE_REV_ACC_CODE: string (nullable = true)
|-- JOB_CONSTRUCTION_VALUE_PCT: float (nullable = true)
|-- JOB_MAX_HOURS: integer (nullable = true)
|-- JOB_CILOC_CODE: string (nullable = true)
|-- JOB_JTR_EXP_FLAG: string (nullable = true)
|-- JOB_IB_EXPENSE_CAT_CODE: string (nullable = true)
|-- JOB_ORIGINAL_CONTRACT_AMT: float (nullable = true)
|-- JOB_DEFAULT_DEPT_CODE: string (nullable = true)
|-- JOB_AP_TAX1_CODE: string (nullable = true)
|-- JOB_AP_TAX2_CODE: string (nullable = true)
|-- JOB_AP_TAX3_CODE: string (nullable = true)
|-- JOB_AR_TAX1_CODE: string (nullable = true)
|-- JOB_AR_TAX2_CODE: string (nullable = true)
|-- JOB_AR_TAX3_CODE: string (nullable = true)
|-- JOB_JB_MAP_CODE: string (nullable = true)
|-- JOB_CAL_SAL_CHARGE_RATE: string (nullable = true)
|-- JOB_WIP_OVERRIDE_CONT_AMT: float (nullable = true)
|-- JOB_TAX1_CODE: string (nullable = true)
|-- JOB_TAX2_CODE: string (nullable = true)
|-- JOB_TAX3_CODE: string (nullable = true)
|-- JOB_APPLY_DB_RULES: string (nullable = true)
|-- JOB_ATTACH_ORASEQ: integer (nullable = true)
|-- JOB_OBJECT_ORASEQ: integer (nullable = true)
|-- JOB_WIP_ROLL_IN_SUBJOB_FLAG: string (nullable = true)
|-- JOB_UE_VALID_FLAG: string (nullable = true)
|-- JOB_PARTN_CODE: string (nullable = true)
|-- JOB_PARTN_TYPE_CODE: string (nullable = true)
|-- JOB_CONTACT_CODE: string (nullable = true)
|-- JOB_ADD_TYPE_CODE: string (nullable = true)
|-- JOB_RULE_CODE: string (nullable = true)
|-- JOB_CREATE_DATE: date (nullable = true)
|-- JOB_PAY_FROM_JOB_COMP_FLAG: string (nullable = true)
|-- JOB_ORIG_BUDGET_EFFECTIVE_DATE: date (nullable = true)
|-- JOB__IU__CREATE_DATE: date (nullable = true)
|-- JOB__IU__CREATE_USER: string (nullable = true)
|-- JOB__IU__UPDATE_DATE: date (nullable = true)
|-- JOB__IU__UPDATE_USER: string (nullable = true)
|-- JOB_COMPLETED_FOR_WIP_FLAG: string (nullable = true)
|-- JOB_EXCLUDE_FROM_WIP_FLAG: string (nullable = true)
|-- JOB_FINALIZE_PROJECTIONS_FLAG: string (nullable = true)
|-- JOB_PW_RATE_CODE: string (nullable = true)
|-- JOB_PW_OV_RATE: float (nullable = true)
|-- JOB_WO_FLAG: string (nullable = true)
|-- JOB_ALLOW_OVERHEAD_FLAG: string (nullable = true)
|-- JOB_DEFAULT_PYOVHD_GRP: string (nullable = true)
|-- JOB_COST_TO_COMPL_OVRD_FLG: string (nullable = true)
|-- JOB_MS_CODE: string (nullable = true)
|-- JOB_SHOW_CPR_AS_COST_AMT_FLAG: string (nullable = true)
|-- JOB_ORIGINAL_FEE_AMT: float (nullable = true)
|-- JOB_PHS_TYPE_REQUIRED_FLG: string (nullable = true)
|-- JOB_BILL_TBL_REV_FCAST_FLG: string (nullable = true)
|-- JOB_SECTOR_CODE: string (nullable = true)
|-- JOB_PAYRATE_SCHEDULE_CODE: string (nullable = true)
|-- JOB_EQP_CHG_NO_RESTART_FLAG: string (nullable = true)
|-- JOB_IGNORE_COSTS_PRIOR_TO_DATE: date (nullable = true)
|-- JOB_JB_RETAINAGE_CODE: string (nullable = true)
|-- JOB_BUDG_OVRD_PROJ_FLAG: string (nullable = true)
|-- JOB_CERTREP_START_DATE: date (nullable = true)
|-- JOB_CERTREP_END_DATE: date (nullable = true)
|-- JOB_CERTREP_SIGNATORY_NAME: string (nullable = true)
|-- JOB_CERTREP_SIGNATORY_TITLE: string (nullable = true)
|-- JOB_CERTREP_CONTRACT_INT: string (nullable = true)
|-- JOB_CERTREP_FRINGE_BEN_PAID: string (nullable = true)
|-- JOB_PROJ_THRESHOLD_PCT: integer (nullable = true)
|-- JOB_PUBLIC_SECTOR_FLAG: string (nullable = true)
|-- JOB_UNIT_PRICED_CONTRACT_FLAG: string (nullable = true)
|-- JOB_FCAST_INCL_SUBJOBS_FLG: string (nullable = true)
|-- JOB_ADJ_BUDG_EQUALS_REV_FLG: string (nullable = true)
|-- JOB_PHS_BUDG_UNITS_FLAG: string (nullable = true)
|-- JOB_PY_OVTRULE_CODE: string (nullable = true)
|-- JOB_ACCT_PARTN_CODE: string (nullable = true)
|-- JOB_ACCT_PARTN_TYPE_CODE: string (nullable = true)
|-- JOB_ACCT_CONTACT_CODE: string (nullable = true)
|-- JOB_ARCHIVE_FLAG: string (nullable = true)
|-- JOB_SUPT_PARTN_CODE: string (nullable = true)
|-- JOB_SUPT_PARTN_TYPE_CODE: string (nullable = true)
|-- JOB_SUPT_CONTACT_CODE: string (nullable = true)
|-- JOB_ASST_PARTN_CODE: string (nullable = true)
|-- JOB_ASST_PARTN_TYPE_CODE: string (nullable = true)
|-- JOB_ASST_CONTACT_CODE: string (nullable = true)
|-- JOB_TENANT_ID: string (nullable = true)
|-- JOB_MAX_CHARGE_PC: float (nullable = true)
|-- JOB_WIP_UNPOSTED_COST_ACC: string (nullable = true)
|-- JOB_WIP_UNPOSTED_COST_DEP: string (nullable = true)
|-- JOB_WIP_UNPOSTED_REV_ACC: string (nullable = true)
|-- JOB_WIP_UNPOSTED_REV_DEP: string (nullable = true)
|-- JOB_WIP_ADJ_ACC_CODE: string (nullable = true)
|-- JOB_WIP_ADJ_DEPT_CODE: string (nullable = true)
|-- JOB_WIP_UNPOSTED_COST_PHASE: string (nullable = true)
|-- JOB_WIP_UNPOSTED_COST_CAT: string (nullable = true)
|-- JOB_WIP_UNPOSTED_REV_PHASE: string (nullable = true)
|-- JOB_WIP_UNPOSTED_REV_CAT: string (nullable = true)
|-- JOB_TRACK_REP_BURDEN: float (nullable = true)
|-- JOB_ALLOC_APDISC_JOB_FLAG: string (nullable = true)
|-- JOB_WIP_LMRG_OVR_FLAG: string (nullable = true)
|-- JOB_WIP_MRG_LOSSPL_DEP: string (nullable = true)
|-- JOB_WIP_MRG_LOSSPL_ACC: string (nullable = true)
|-- JOB_WIP_MRG_LOSSBS_DEP: string (nullable = true)
|-- JOB_WIP_MRG_LOSSBS_ACC: string (nullable = true)
|-- JOB_WIP_OH_LOSSPL_DEP: string (nullable = true)
|-- JOB_WIP_OH_LOSSPL_ACC: string (nullable = true)
|-- JOB_WIP_OH_LOSSBS_DEP: string (nullable = true)
|-- JOB_WIP_OH_LOSSBS_ACC: string (nullable = true)
|-- JOB_ALLOW_ETMSHT_ENTRIES_FLAG: string (nullable = true)
|-- JOB_LAST_SESSIONINFO: string (nullable = true)
|-- JOB_EXCL_REV_BUDG_WIP: string (nullable = true)
|-- JOB_EXCL_BILLING_WIP: string (nullable = true)
|-- JOB_ORIG_SESSIONINFO: string (nullable = true)
|-- JOB_EXCL_CONT_AMT_WIP_FLAG: string (nullable = true)
|-- JOB_SICK_POLICY_CODE: string (nullable = true)
|-- JOB_PYRL_BILL_RATE_FLG: string (nullable = true)
|-- JOB_REPLACEMENT_VALUE_CAP_PCT: integer (nullable = true)
|-- JOB_DISCOUNT_PCT: integer (nullable = true)
|-- JOB_VALUE_CAP_CUTOFF_AMT: float (nullable = true)
|-- JOB_UNI_LOC_CODE: string (nullable = true)
|-- JOB_CERTREP_CONTRACT_ID_ODOT: string (nullable = true)
|-- JOB_CERTREP_PROJECT_ID_ODOT: string (nullable = true)
|-- JOB_OT_DOT_CHECKS: string (nullable = true)
|-- JOB_MAIN_JOB_WIP_COMB: string (nullable = true)
|-- JOB_FINAL_POST_DEPT_WIP_COMB: string (nullable = true)
|-- JOB_EXCL_FROM_CHG_CAP_FLAG: string (nullable = true)
|-- JOB_ALLOW_PCI_ENTRY_FLAG: string (nullable = true)
|-- JOB_JMA_CODE: string (nullable = true)
|-- JOB_PAYROLL_RATE_POLICY: string (nullable = true)
|-- JOB_PY_PCM_MAP_CODE: string (nullable = true)
|-- JOB_PY_PIT_TEMPLATE_CODE: string (nullable = true)
|-- JOB_PHS_TEMPLATE_CODE: string (nullable = true)
|-- JOB_APPLY_PHS_TEMPLATE_FLAG: string (nullable = true)
|-- JOB_V_UUID: string (nullable = true)
|-- JOB_SHOW_COST_AMT_WIP_FLAG: string (nullable = true)
|-- JOB_CERTREP_EMAIL: string (nullable = true)
|-- Load_TimeStamp: timestamp (nullable = true)
|-- Source_Table: string (nullable = true)
Inferred schema:
root
|-- JOB_COMP_CODE: string (nullable = true)
|-- JOB_CODE: string (nullable = true)
|-- JOB_CTRL_CODE: string (nullable = true)
|-- JOB_NAME: string (nullable = true)
|-- JOB_CUST_CODE: string (nullable = true)
|-- JOB_CONTRACT_CODE: string (nullable = true)
|-- JOB_WIP_DEPT_CODE: string (nullable = true)
|-- JOB_WIP_ACC_CODE: string (nullable = true)
|-- JOB_LBC_DEPT_CODE: string (nullable = true)
|-- JOB_LBC_ACC_CODE: string (nullable = true)
|-- JOB_LTC_DEPT_CODE: string (nullable = true)
|-- JOB_LTC_ACC_CODE: string (nullable = true)
|-- JOB_CC_DEPT_CODE: string (nullable = true)
|-- JOB_CC_ACC_CODE: string (nullable = true)
|-- JOB_BILL_DEPT_CODE: string (nullable = true)
|-- JOB_BILL_ACC_CODE: string (nullable = true)
|-- JOB_BILL_AMT: float (nullable = true)
|-- JOB_COST_METH_CODE: string (nullable = true)
|-- JOB_STATUS_CODE: string (nullable = true)
|-- JOB_ACCMETH_CODE: string (nullable = true)
|-- JOB_CERTIFY_CODE: string (nullable = true)
|-- JOB_SIZE_CODE: string (nullable = true)
|-- JOB_WM_CODE: string (nullable = true)
|-- JOB_ACTION_CODE: string (nullable = true)
|-- JOB_EST_START_DATE: date (nullable = true)
|-- JOB_ACT_START_DATE: date (nullable = true)
|-- JOB_EST_COMPL_DATE: date (nullable = true)
|-- JOB_ACT_COMPL_DATE: date (nullable = true)
|-- JOB_LST_ADDON_DATE: date (nullable = true)
|-- JOB_LST_REC_DATE: date (nullable = true)
|-- JOB_REC_AMT: float (nullable = true)
|-- JOB_HB_REC_AMT: float (nullable = true)
|-- JOB_REVREC_AMT: float (nullable = true)
|-- JOB_HB_AMT: float (nullable = true)
|-- JOB_PROFREC_AMT: float (nullable = true)
|-- JOB_LST_REC_PC: float (nullable = true)
|-- JOB_LOC_CODE: string (nullable = true)
|-- JOB_BUDG_UNIT: float (nullable = true)
|-- JOB_COMPL_UNIT: float (nullable = true)
|-- JOB_DISB_AMT: float (nullable = true)
|-- JOB_REVREC_LST_PC: float (nullable = true)
|-- JOB_REVREC_PC: float (nullable = true)
|-- JOB_CONTRACT_AMT: float (nullable = true)
|-- JOB_COST_FLAG: string (nullable = true)
|-- JOB_BILL_FLAG: string (nullable = true)
|-- JOB_SUB_FLAG: string (nullable = true)
|-- JOB_SEC_GROUP: integer (nullable = true)
|-- JOB_BUDGCST_SAME_LEVEL_FLAG: string (nullable = true)
|-- JOB_WBSV_CODE1: string (nullable = true)
|-- JOB_WBSV_CODE2: string (nullable = true)
|-- JOB_WBSV_CODE3: string (nullable = true)
|-- JOB_WBSV_CODE4: string (nullable = true)
|-- JOB_WBSV_CODE5: string (nullable = true)
|-- JOB_WBSV_CODE6: string (nullable = true)
|-- JOB_WBSV_CODE7: string (nullable = true)
|-- JOB_WBSV_CODE8: string (nullable = true)
|-- JOB_WBSV_CODE9: string (nullable = true)
|-- JOB_WBSV_CODE10: string (nullable = true)
|-- JOB_WBSV_CODE11: string (nullable = true)
|-- JOB_WBSV_CODE12: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG1: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG2: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG3: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG4: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG5: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG6: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG7: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG8: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG9: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG10: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG11: string (nullable = true)
|-- JOB_WBSV_REQUIRED_FLAG12: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG1: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG2: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG3: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG4: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG5: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG6: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG7: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG8: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG9: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG10: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG11: string (nullable = true)
|-- JOB_WBSV_EDITABLE_FLAG12: string (nullable = true)
|-- JOB_BID_FLAG: string (nullable = true)
|-- JOB_BID_STATUS_CODE: string (nullable = true)
|-- JOB_BID_CODE: string (nullable = true)
|-- JOB_MAKEUP_FLAG: string (nullable = true)
|-- JOB_PM_FLAG: string (nullable = true)
|-- JOB_BILL_METH_CODE: string (nullable = true)
|-- JOB_INV_FORMAT_CODE: string (nullable = true)
|-- JOB_BID_SUBMIT_DATE: date (nullable = true)
|-- JOB_PROPERTY_ID: integer (nullable = true)
|-- JOB_AREA_DISTRICT: string (nullable = true)
|-- JOB_PROVINCE_CODE: string (nullable = true)
|-- JOB_REVENUE_GEN_STATE: string (nullable = true)
|-- JOB_UNBILLED_REV_DEPT_CODE: string (nullable = true)
|-- JOB_UNBILLED_REV_ACC_CODE: string (nullable = true)
|-- JOB_HIER: string (nullable = true)
|-- JOB_CHG_SEQ_NUM: integer (nullable = true)
|-- JOB_BUDGR_SEQ_NUM: integer (nullable = true)
|-- JOB_AUTH_RQ_SEQ_NUM: integer (nullable = true)
|-- JOB_SI_SEQ_NUM: integer (nullable = true)
|-- JOB_REVREC_CURR_DATE: date (nullable = true)
|-- JOB_REVREC_LST_AMT: float (nullable = true)
|-- WORK_LOC: string (nullable = true)
|-- JOB_MUTLI_OVHD_PC_FLAG: string (nullable = true)
|-- JOB_WORK_LOC: string (nullable = true)
|-- JOB_POLICY_NO: string (nullable = true)
|-- JOB_PREVAILING_WAGE: string (nullable = true)
|-- JOB_PL_POLICY_NO: string (nullable = true)
|-- JOB_FULLY_PAID_INVS: integer (nullable = true)
|-- JOB_DAYS_OUTST_INV_PAID_TTL: integer (nullable = true)
|-- JOB_RATE_BY_JOB_FLAG: string (nullable = true)
|-- JOB_USE_PAY_BILL_RATE_FLAG: string (nullable = true)
|-- JOB_USE_EQP_BILL_RATE_FLAG: string (nullable = true)
|-- JOB_CUST_CONTACT_NAME: string (nullable = true)
|-- JOB_TERM_CODE: string (nullable = true)
|-- JOB_INVOICE_GROUP_CODE: string (nullable = true)
|-- JOB_BILLING_TYPE_CODE: string (nullable = true)
|-- JOB_INVOICE_FORMAT_CODE: string (nullable = true)
|-- JOB_CONSTRUCTION_VALUE: float (nullable = true)
|-- JOB_MAX_HOURLY_RATE: float (nullable = true)
|-- JOB_MAX_BILLING_AMT: float (nullable = true)
|-- JOB_MAX_BILLING_BUDGET_AMT: float (nullable = true)
|-- JOB_BILLING_RATE_TABLE_CODE: string (nullable = true)
|-- JOB_IB_ALLOW_FLAG: string (nullable = true)
|-- JOB_IB_FULL_TARIFF_FLAG: string (nullable = true)
|-- JOB_CONT_TYPE_CODE: string (nullable = true)
|-- JOB_LONG_CODE: string (nullable = true)
|-- JOB_RESERVE_REV_DEPT_CODE: string (nullable = true)
|-- JOB_RESERVE_REV_ACC_CODE: string (nullable = true)
|-- JOB_CONSTRUCTION_VALUE_PCT: float (nullable = true)
|-- JOB_MAX_HOURS: integer (nullable = true)
|-- JOB_CILOC_CODE: string (nullable = true)
|-- JOB_JTR_EXP_FLAG: string (nullable = true)
|-- JOB_IB_EXPENSE_CAT_CODE: string (nullable = true)
|-- JOB_ORIGINAL_CONTRACT_AMT: float (nullable = true)
|-- JOB_DEFAULT_DEPT_CODE: string (nullable = true)
|-- JOB_AP_TAX1_CODE: string (nullable = true)
|-- JOB_AP_TAX2_CODE: string (nullable = true)
|-- JOB_AP_TAX3_CODE: string (nullable = true)
|-- JOB_AR_TAX1_CODE: string (nullable = true)
|-- JOB_AR_TAX2_CODE: string (nullable = true)
|-- JOB_AR_TAX3_CODE: string (nullable = true)
|-- JOB_JB_MAP_CODE: string (nullable = true)
|-- JOB_CAL_SAL_CHARGE_RATE: string (nullable = true)
|-- JOB_WIP_OVERRIDE_CONT_AMT: float (nullable = true)
|-- JOB_TAX1_CODE: string (nullable = true)
|-- JOB_TAX2_CODE: string (nullable = true)
|-- JOB_TAX3_CODE: string (nullable = true)
|-- JOB_APPLY_DB_RULES: string (nullable = true)
|-- JOB_ATTACH_ORASEQ: integer (nullable = true)
|-- JOB_OBJECT_ORASEQ: integer (nullable = true)
|-- JOB_WIP_ROLL_IN_SUBJOB_FLAG: string (nullable = true)
|-- JOB_UE_VALID_FLAG: string (nullable = true)
|-- JOB_PARTN_CODE: string (nullable = true)
|-- JOB_PARTN_TYPE_CODE: string (nullable = true)
|-- JOB_CONTACT_CODE: string (nullable = true)
|-- JOB_ADD_TYPE_CODE: string (nullable = true)
|-- JOB_RULE_CODE: string (nullable = true)
|-- JOB_CREATE_DATE: date (nullable = true)
|-- JOB_PAY_FROM_JOB_COMP_FLAG: string (nullable = true)
|-- JOB_ORIG_BUDGET_EFFECTIVE_DATE: date (nullable = true)
|-- JOB__IU__CREATE_DATE: date (nullable = true)
|-- JOB__IU__CREATE_USER: string (nullable = true)
|-- JOB__IU__UPDATE_DATE: date (nullable = true)
|-- JOB__IU__UPDATE_USER: string (nullable = true)
|-- JOB_COMPLETED_FOR_WIP_FLAG: string (nullable = true)
|-- JOB_EXCLUDE_FROM_WIP_FLAG: string (nullable = true)
|-- JOB_FINALIZE_PROJECTIONS_FLAG: string (nullable = true)
|-- JOB_PW_RATE_CODE: string (nullable = true)
|-- JOB_PW_OV_RATE: float (nullable = true)
|-- JOB_WO_FLAG: string (nullable = true)
|-- JOB_ALLOW_OVERHEAD_FLAG: string (nullable = true)
|-- JOB_DEFAULT_PYOVHD_GRP: string (nullable = true)
|-- JOB_COST_TO_COMPL_OVRD_FLG: string (nullable = true)
|-- JOB_MS_CODE: string (nullable = true)
|-- JOB_SHOW_CPR_AS_COST_AMT_FLAG: string (nullable = true)
|-- JOB_ORIGINAL_FEE_AMT: float (nullable = true)
|-- JOB_PHS_TYPE_REQUIRED_FLG: string (nullable = true)
|-- JOB_BILL_TBL_REV_FCAST_FLG: string (nullable = true)
|-- JOB_SECTOR_CODE: string (nullable = true)
|-- JOB_PAYRATE_SCHEDULE_CODE: string (nullable = true)
|-- JOB_EQP_CHG_NO_RESTART_FLAG: string (nullable = true)
|-- JOB_IGNORE_COSTS_PRIOR_TO_DATE: date (nullable = true)
|-- JOB_JB_RETAINAGE_CODE: string (nullable = true)
|-- JOB_BUDG_OVRD_PROJ_FLAG: string (nullable = true)
|-- JOB_CERTREP_START_DATE: date (nullable = true)
|-- JOB_CERTREP_END_DATE: date (nullable = true)
|-- JOB_CERTREP_SIGNATORY_NAME: string (nullable = true)
|-- JOB_CERTREP_SIGNATORY_TITLE: string (nullable = true)
|-- JOB_CERTREP_CONTRACT_NUMBER: string (nullable = true)
|-- JOB_CERTREP_FRINGE_BEN_PAID: string (nullable = true)
|-- JOB_PROJ_THRESHOLD_PCT: integer (nullable = true)
|-- JOB_PUBLIC_SECTOR_FLAG: string (nullable = true)
|-- JOB_UNIT_PRICED_CONTRACT_FLAG: string (nullable = true)
|-- JOB_FCAST_INCL_SUBJOBS_FLG: string (nullable = true)
|-- JOB_ADJ_BUDG_EQUALS_REV_FLG: string (nullable = true)
|-- JOB_PHS_BUDG_UNITS_FLAG: string (nullable = true)
|-- JOB_PY_OVTRULE_CODE: string (nullable = true)
|-- JOB_ACCT_PARTN_CODE: string (nullable = true)
|-- JOB_ACCT_PARTN_TYPE_CODE: string (nullable = true)
|-- JOB_ACCT_CONTACT_CODE: string (nullable = true)
|-- JOB_ARCHIVE_FLAG: string (nullable = true)
|-- JOB_SUPT_PARTN_CODE: string (nullable = true)
|-- JOB_SUPT_PARTN_TYPE_CODE: string (nullable = true)
|-- JOB_SUPT_CONTACT_CODE: string (nullable = true)
|-- JOB_ASST_PARTN_CODE: string (nullable = true)
|-- JOB_ASST_PARTN_TYPE_CODE: string (nullable = true)
|-- JOB_ASST_CONTACT_CODE: string (nullable = true)
|-- JOB_TENANT_ID: string (nullable = true)
|-- JOB_MAX_CHARGE_PC: float (nullable = true)
|-- JOB_WIP_UNPOSTED_COST_ACC: string (nullable = true)
|-- JOB_WIP_UNPOSTED_COST_DEP: string (nullable = true)
|-- JOB_WIP_UNPOSTED_REV_ACC: string (nullable = true)
|-- JOB_WIP_UNPOSTED_REV_DEP: string (nullable = true)
|-- JOB_WIP_ADJ_ACC_CODE: string (nullable = true)
|-- JOB_WIP_ADJ_DEPT_CODE: string (nullable = true)
|-- JOB_WIP_UNPOSTED_COST_PHASE: string (nullable = true)
|-- JOB_WIP_UNPOSTED_COST_CAT: string (nullable = true)
|-- JOB_WIP_UNPOSTED_REV_PHASE: string (nullable = true)
|-- JOB_WIP_UNPOSTED_REV_CAT: string (nullable = true)
|-- JOB_TRACK_REP_BURDEN: float (nullable = true)
|-- JOB_ALLOC_APDISC_JOB_FLAG: string (nullable = true)
|-- JOB_WIP_LMRG_OVR_FLAG: string (nullable = true)
|-- JOB_WIP_MRG_LOSSPL_DEP: string (nullable = true)
|-- JOB_WIP_MRG_LOSSPL_ACC: string (nullable = true)
|-- JOB_WIP_MRG_LOSSBS_DEP: string (nullable = true)
|-- JOB_WIP_MRG_LOSSBS_ACC: string (nullable = true)
|-- JOB_WIP_OH_LOSSPL_DEP: string (nullable = true)
|-- JOB_WIP_OH_LOSSPL_ACC: string (nullable = true)
|-- JOB_WIP_OH_LOSSBS_DEP: string (nullable = true)
|-- JOB_WIP_OH_LOSSBS_ACC: string (nullable = true)
|-- JOB_ALLOW_ETMSHT_ENTRIES_FLAG: string (nullable = true)
|-- JOB_LAST_SESSIONINFO: string (nullable = true)
|-- JOB_EXCL_REV_BUDG_WIP: string (nullable = true)
|-- JOB_EXCL_BILLING_WIP: string (nullable = true)
|-- JOB_ORIG_SESSIONINFO: string (nullable = true)
|-- JOB_EXCL_CONT_AMT_WIP_FLAG: string (nullable = true)
|-- JOB_SICK_POLICY_CODE: string (nullable = true)
|-- JOB_PYRL_BILL_RATE_FLG: string (nullable = true)
|-- JOB_REPLACEMENT_VALUE_CAP_PCT: integer (nullable = true)
|-- JOB_DISCOUNT_PCT: integer (nullable = true)
|-- JOB_VALUE_CAP_CUTOFF_AMT: float (nullable = true)
|-- JOB_UNI_LOC_CODE: string (nullable = true)
|-- JOB_CERTREP_CONTRACT_ID_ODOT: string (nullable = true)
|-- JOB_CERTREP_PROJECT_ID_ODOT: string (nullable = true)
|-- JOB_OT_DOT_CHECKS: string (nullable = true)
|-- JOB_MAIN_JOB_WIP_COMB: string (nullable = true)
|-- JOB_FINAL_POST_DEPT_WIP_COMB: string (nullable = true)
|-- JOB_EXCL_FROM_CHG_CAP_FLAG: string (nullable = true)
|-- JOB_ALLOW_PCI_ENTRY_FLAG: string (nullable = true)
|-- JOB_JMA_CODE: string (nullable = true)
|-- JOB_PAYROLL_RATE_POLICY: string (nullable = true)
|-- JOB_PY_PCM_MAP_CODE: string (nullable = true)
|-- JOB_PY_PIT_TEMPLATE_CODE: string (nullable = true)
|-- JOB_PHS_TEMPLATE_CODE: string (nullable = true)
|-- JOB_APPLY_PHS_TEMPLATE_FLAG: string (nullable = true)
|-- JOB_V_UUID: string (nullable = true)
|-- JOB_SHOW_COST_AMT_WIP_FLAG: string (nullable = true)
|-- JOB_CERTREP_EMAIL: string (nullable = true)
|-- Load_TimeStamp: timestamp (nullable = true)
|-- Source_Table: string (nullable = true)
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
3 hours ago - last edited 3 hours ago
Hi @Svish ,
You have one line that differs:
JOB_CERTREP_CONTRACT_INT: string (nullable = true) vs.
JOB_CERTREP_CONTRACT_NUMBER: string (nullable = true)
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group
Announcements
Related Content
- Error when trying to run model in staging in Data Engineering
- Mock user doesn't have the required privileges to access catalog `remorph` while running 'make test' in Data Engineering
- Getting error while using Live.target_table in dlt pipeline in Data Engineering
- Old files also getting added in dlt autoloader in Data Engineering