Skip to content

check modifications #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bruno-ariano merged 1 commit into from
Feb 16, 2026
Merged

check modifications #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions conf/base.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
process {
conda = "/ssu/gassu/conda_envs/tdbsumstat"
container = "ghcr.io/bruno-ariano/tdbsumstat:1.0"
//conda = "/software/cardinal_analysis/ht/conda_envs/tdbsumstat"
//conda = "/ssu/gassu/conda_envs/tdbsumstat"
//container = "ghcr.io/bruno-ariano/tdbsumstat:1.0"
conda = "/software/cardinal_analysis/ht/conda_envs/tdbsumstat"
// conda = "${projectDir}/pipeline_environment.yml"
Comment on lines +4 to 5

Copilot AI Feb 16, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conf/base.config is included by default, so hardcoding an absolute, institution-specific conda path (and commenting out the container) will break portability for other users/runners. Consider reverting to a repo-relative env file (e.g., ${projectDir}/pipeline_environment.yml) or leaving conda/container unset here and configuring them via profiles or a local config override.

Suggested change
conda = "/software/cardinal_analysis/ht/conda_envs/tdbsumstat"
// conda = "${projectDir}/pipeline_environment.yml"
// conda = "/software/cardinal_analysis/ht/conda_envs/tdbsumstat"
conda = "${projectDir}/pipeline_environment.yml"

Copilot uses AI. Check for mistakes.

// memory errors which should be retried. otherwise error out
Expand Down
4 changes: 2 additions & 2 deletions modules/ingestion/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ process INGEST_DATA {
script:
def qc = params.qc ? "--qc" : ""
def pvar_file = params.pvar_file ? "--pvar-file ${params.pvar_file}" : ""
def pvar_file = params.permuted ? "--permuted" : ""
def permuted = params.permuted ? "--permuted" : ""
"""
tdbsumstat ingest \
--uri-path TileDB_${params.tiledb_name}\
--file-path ${list_files} \
--mapping-file ${mapping_file} \
--type-sumstat ${params.type_sumstat} \
--mac ${params.mac} ${qc} ${pvar_file} ${params.permuted}
--mac ${params.mac} ${qc} ${pvar_file} ${permuted}
# Rename the metadata parts directory to include the list_files name for uniqueness
if [ -d "TileDB_${params.tiledb_name}_metadata_parts" ]; then
mv "TileDB_${params.tiledb_name}_metadata_parts" "TileDB_${params.tiledb_name}_metadata_parts_${list_files.name}"
Expand Down
2 changes: 1 addition & 1 deletion tdbsumstat/cli/ingestion.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ def ingest(uri_path:str,
chunk_pl = pl.read_csv(file,separator=sep,low_memory=True ,null_values="NA")
harmonized_object.harmonize(sumstat = chunk_pl, trait = trait, cell = cell,
gene = gene, pheno_var = pheno_var, n = n, n_cases = n_cases,
n_controls = n_controls, mac = mac, permuted = permuted)
n_controls = n_controls, mac = mac)
#Performing QC using GWASLAB
if qc:
harmonized_object.qc_sumstat(file_path = file)
Expand Down
28 changes: 15 additions & 13 deletions tdbsumstat/utils/harmonize_ingest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ class HarmonizationError(Exception):


class Harmonize:
def __init__(self, mapping_file: str, uri: str, type_sumstat: str, pvar_file: str, type_trait: str, mac: int, permuted: False):
def __init__(self, mapping_file: str, uri: str, type_sumstat: str, pvar_file: str, type_trait: str, mac: int, permuted: bool):
self.mapping_file = mapping_file
self.uri = uri
self.pvar_file = pvar_file
Expand All @@ -33,6 +33,7 @@ def __init__(self, mapping_file: str, uri: str, type_sumstat: str, pvar_file: st
self.dimension_tiledb = []
self.mac = mac
self.permuted = permuted
print(self.permuted)

Copilot AI Feb 16, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the print(self.permuted) debug statement from __init__. This will write to stdout for every ingestion run and can interfere with structured logs / CLI output; use the existing logger (or remove entirely) if you need to surface this value.

Suggested change
print(self.permuted)
logger.debug("Harmonize created with permuted=%s", self.permuted)

Copilot uses AI. Check for mistakes.

def create_mapping(self):
df = pd.read_csv(self.mapping_file, header=None, names=["key", "value"])
Expand Down Expand Up @@ -271,24 +272,25 @@ def harmonize(self,
(10 ** (-pl.col("LOG10P"))).alias("P")
)

#Calculate p-value from z-score
self.chunk_pl = self.chunk_pl.drop('P')
if self.permuted==False:
self.chunk_pl = self.chunk_pl.with_columns(
(pl.col("BETA") / pl.col("SE")).pow(2).map_batches(
lambda x: pl.Series(stats.chi2.sf(x.to_numpy(), df=1)),
return_dtype=pl.Float64
).alias('P')
)
else:

if self.permuted:
self.chunk_pl = self.chunk_pl.with_columns(
(
pl.col("BETA").pow(2) /
pl.col("P").map_batches(
lambda x: pl.Series(stats.chi2.isf(x.to_numpy(), df=1))

Copilot AI Feb 16, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the permuted branch, map_batches(...) does not specify return_dtype. Polars may infer an Object dtype or warn/error depending on execution context; set an explicit float dtype (consistent with the non-permuted branch) to keep the SE computation stable.

Suggested change
lambda x: pl.Series(stats.chi2.isf(x.to_numpy(), df=1))
lambda x: pl.Series(stats.chi2.isf(x.to_numpy(), df=1)),
return_dtype=pl.Float64

Copilot uses AI. Check for mistakes.
)
).sqrt().alias("SE")
)
).sqrt().alias("SE"))

else:
#Calculate p-value from z-score
self.chunk_pl = self.chunk_pl.drop('P')

Copilot AI Feb 16, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.chunk_pl.drop('P') will raise if the input summary stats do not already contain a P column (e.g., when only BETA/SE are provided). If the intent is to recompute P-values, either drop with strict=False or conditionally drop only when P exists.

Suggested change
self.chunk_pl = self.chunk_pl.drop('P')
if "P" in self.chunk_pl.columns:
self.chunk_pl = self.chunk_pl.drop('P')

Copilot uses AI. Check for mistakes.
self.chunk_pl = self.chunk_pl.with_columns(
(pl.col("BETA") / pl.col("SE")).pow(2).map_batches(
lambda x: pl.Series(stats.chi2.sf(x.to_numpy(), df=1)),
return_dtype=pl.Float64
).alias('P')
)

def qc_sumstat(self, file_path:str):
directory = self.uri + "_logs"
Expand Down
Loading