Skip to content

Owo Sugiana / payment-report

Go to a project
Commit 1555f0a1 by Owo Sugiana
Options

Class yang tidak bergantung pada command line argument

1 parent 396ca087
Inline Side-by-side
Showing 22 changed files with 457 additions and 286 deletions
CHANGES.txt
3.0 2023-03-27
--------------
- Class yang tidak bergantung pada command line argument
2.13 2023-03-18
---------------
- Tambah Fortuna
......
README.rst
......@@ -36,17 +36,8 @@ diperlukan user ``report``::
GRANT SELECT ON pad_payment TO report;
GRANT SELECT ON pad_reversal TO report;
Lalu buat file konfigurasi bernama ``pad-report.ini``::
[main]
models = opensipkd.pad.models.default
service = opensipkd.pad.services.default
db_url = postgresql://report:password@localhost:5432/simpad
report_db_url = postgresql://report:password@localhost:5432/payment_report
pid_file = /home/sugiana/tmp/pad-report.pid
log_file = /home/sugiana/log/pad-report.log
Jika ada database VA / QRIS maka tambahkan::
File konfigurasinya lihat di ``pad.ini``. Jika ada database VA / QRIS maka
tambahkan::
va_db_url = postgresql://report:password@localhost:5432/va_bjb
......@@ -57,7 +48,7 @@ Buat tabel yang dibutuhkan di database ``payment_report``::
Kemudian buat file ``/etc/cron.d/payment-report`` berikut ini::
MAILTO=""
*/5 * * * * sugiana cd /home/sugiana && env/bin/pad_report pad-report.ini
*/5 * * * * sugiana cd /home/sugiana && env/bin/payment_report pad-report.ini
Lalu pantau log-nya. Jika sudah ada penampakan lihat hasilnya di tabel ``pad_report``.
......@@ -78,22 +69,14 @@ Berikan hak akses yang diperlukan user ``report``::
GRANT SELECT ON bphtb.bphtb_ppat TO report;
GRANT SELECT ON log_iso TO report;
Lalu buat file konfigurasi bernama ``bphtb-report.ini``::
[main]
models = opensipkd.bphtb.models.default
db_url = postgresql://report:password@localhost:5434/bphtb
report_db_url = postgresql://report:password@localhost:5434/payment_report
pid_file = /home/sugiana/tmp/bphtb-report.pid
log_file = /home/sugiana/log/bphtb-report.log
Jika ada database VA / QRIS maka tambahkan::
File konfigurasi lihat di ``bphtb.ini``. Jika ada database VA / QRIS maka
tambahkan::
va_db_url = postgresql://report:password@localhost:5432/va_bjb
Kemudian di file ``/etc/cron.d/payment-report`` tambahkan::
*/5 * * * * sugiana cd /home/sugiana && env/bin/bphtb_report bphtb-report.ini
*/5 * * * * sugiana cd /home/sugiana && env/bin/payment_report bphtb-report.ini
Lalu pantau log-nya. Jika sudah ada penampakan lihat hasilnya di tabel ``bphtb_report``.
......@@ -114,37 +97,14 @@ Berikan hak akses yang diperlukan user ``report``::
GRANT SELECT ON log_iso TO report;
CREATE INDEX log_iso_payment ON log_iso (mti, bit_003, bit_039, bit_061);
Lalu buat file konfigurasi bernama ``pbb-report.ini``::
[main]
models = sismiop.models.default
db_url = postgresql://report:password@localhost:5434/pbb
report_db_url = postgresql://report:password@localhost:5434/payment_report
pid_file = /home/sugiana/tmp/pbb-report.pid
log_file = /home/sugiana/log/pbb-report.log
Jika ada database VA / QRIS maka tambahkan::
File konfigurasi lihat di ``pbb.ini``. Jika ada database VA / QRIS maka
tambahkan::
va_db_url = postgresql://report:password@localhost:5432/va_bjb
Kemudian di file ``/etc/cron.d/payment-report`` tambahkan::
*/5 * * * * sugiana cd /home/sugiana && env/bin/pbb_report pbb-report.ini
*/5 * * * * sugiana cd /home/sugiana && env/bin/payment_report pbb-report.ini
Lalu pantau log-nya. Jika sudah ada penampakan lihat hasilnya di tabel ``pbb_report``.
Database Sybase
---------------
Bila sumber data berasal dari Sybase maka kita bisa gunakan driver FreeTDS::
$ sudo apt-get install freetds-dev freetds-bin unixodbc-dev tdsodbc
$ sudo dpkg-reconfigure tdsodbc
Nanti ada pertanyaan apakah FreeTDS akan dipasang sebagai driver ODBC. Jawab
Ya. Kemudian pasang modul Python-nya::
$ ~/env/bin/pip install -r sybase-requirements.txt
Selanjutnya lihat ``pbb-odbc.ini``.
bphtb.ini
[main]
module = payment_report.bphtb.default
models = opensipkd.bphtb.models.default
db_url = postgresql://user:pass@localhost/db
#va_db_url = postgresql://user:pass@localhost/db
report_db_url = postgresql://user:pass@localhost/db
pid_file = /home/sugiana/tmp/bphtb-report.pid
log_file = /home/sugiana/log/bphtb-report.log
[formatter_generic]
format = %(asctime)s %(levelname)s %(name)s %(message)s
[formatter_simple]
format = %(asctime)s %(levelname)s %(message)s
[handler_console]
class = StreamHandler
stream = sys.stdout
formatter = generic
[handler_file]
class = FileHandler
filename = /home/sugiana/log/bphtb-report.log
formatter = simple
[logger_root]
handlers = console, file
level = INFO
pad.ini
[main]
moduel = payment_report.pad.default
models = opensipkd.pad.models.default
service = opensipkd.pad.services.default
db_url = postgresql://user:pass@localhost/db
......@@ -6,3 +7,23 @@ db_url = postgresql://user:pass@localhost/db
report_db_url = postgresql://user:pass@localhost/db
pid_file = /home/sugiana/tmp/pad-report.pid
log_file = /home/sugiana/log/pad-report.log
[formatter_generic]
format = %(asctime)s %(levelname)s %(name)s %(message)s
[formatter_simple]
format = %(asctime)s %(levelname)s %(message)s
[handler_console]
class = StreamHandler
stream = sys.stdout
formatter = generic
[handler_file]
class = FileHandler
filename = /home/sugiana/log/pad-report.log
formatter = simple
[logger_root]
handlers = console, file
level = INFO
payment_report/scripts/bphtb.py → payment_report/bphtb/default.py
import sys
import pdb
from datetime import (
datetime,
timedelta,
date,
)
from ISO8583.ISOErrors import BitNotSet
......@@ -27,6 +24,7 @@ from sqlalchemy.exc import (
)
from opensipkd.string import FixLength
from opensipkd.waktu import dmyhms
from opensipkd.bphtb.models.default import Invoice
from opensipkd.bphtb.models.customer import CustomerMixin
from opensipkd.bphtb.models.perolehan import PerolehanMixin
from opensipkd.bphtb.models.invoice import InvoiceMixin
......@@ -39,7 +37,6 @@ from opensipkd.bphtb.models.default import (
)
from opensipkd.bphtb.structure import NOP
from opensipkd.bphtb.services.default.structure import INVOICE_ID
from iso8583_web.models.meta import Base as BaseConf
from opensipkd.iso8583.bjb.bphtb import Doc
from opensipkd.iso8583.bjb.bphtb.structure import PAYMENT_CODE
from opensipkd.iso8583.bjb.models import Log as JsonLog
......@@ -48,13 +45,12 @@ from ..models import (
Base,
Bphtb,
)
from .common import (
from ..common import (
get_iso,
get_channel_name_by_row,
get_channel_name,
get_channel_info_by_iso,
BaseApp,
init_db as base_init_db,
one_day,
InvalidSource,
BANK_NAMES,
......@@ -172,8 +168,6 @@ class App(BaseApp):
def __init__(self, argv):
super().__init__(argv)
if not self.pid:
return
self.base_q_func = self.prod_session.query(func.count())
self.is_sukabumi_kota = False
self.is_cartenz = False
......@@ -241,14 +235,12 @@ class App(BaseApp):
self.base_q_iso_payment = self.prod_session.query(IsoPayment)
try:
self.base_q_iso_payment.first()
except ProgrammingError:
except (ProgrammingError, OperationalError):
self.prod_session.rollback()
self.base_q_iso_payment = self.prod_session.query(
AlternativeIsoPayment)
def set_default_models(self):
if self.models:
Invoice = self.models.Invoice
self.base_q_inv = self.prod_session.query(Invoice)
self.base_q_inv.first()
self.Customer = Customer
......@@ -458,7 +450,8 @@ class App(BaseApp):
op_alamat=inv.alamat_op, npop=inv.npop, bumi_luas=inv.luas_tanah,
bng_luas=inv.luas_bangunan, nilai_bphtb=pay.jumlah_yg_dibayar,
jenis_perolehan=str(inv.jns_perolehan_hak), ppat=inv.nama_notaris,
channel_id=channel_id, channel_nama=channel_nama)
channel_id=channel_id, channel_nama=channel_nama,
status_pembayaran=inv.status_pembayaran)
def create_data_cartenz(self, pay):
self.invoice_id = cartenz_invoice_id(pay)
......@@ -537,13 +530,3 @@ class App(BaseApp):
nilai_bphtb=pay.bayar, jenis_perolehan=perolehan.nama,
ppat=cust.nama.strip(), channel_id=channel_id,
channel_nama=channel_nama)
def main(argv=sys.argv[1:]):
app = App(argv)
if app.pid:
app.run()
def init_db(argv=sys.argv[1:]):
base_init_db(Base.metadata, BaseConf.metadata, argv)
payment_report/scripts/common.py → payment_report/common.py
......@@ -3,6 +3,7 @@ import os
import logging
import csv
from time import time
from logging import getLogger
from datetime import (
date,
datetime,
......@@ -23,7 +24,7 @@ import transaction
from zope.sqlalchemy import register
from opensipkd.views.models import Conf as BaseConf
from opensipkd.iso8583.bjb.scripts.common import get_module_object
from ..models import Base
from .models import Base
from .tools import (
plain_values,
update,
......@@ -102,7 +103,7 @@ def humanize_time(secs):
def get_file(filename):
base_dir = os.path.split(__file__)[0]
fullpath = os.path.join(base_dir, 'data', filename)
fullpath = os.path.join(base_dir, 'scripts', 'data', filename)
return open(fullpath)
......@@ -206,32 +207,6 @@ def get_keys(iso):
return d
def create_log(log_file):
file_handler = logging.FileHandler(log_file)
file_handler.setFormatter(formatter)
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setFormatter(formatter)
log = logging.getLogger(sys.argv[0])
log.setLevel(logging.DEBUG)
log.addHandler(file_handler)
log.addHandler(console_handler)
return log
def get_parser():
pars = ArgumentParser()
pars.add_argument('conf')
pars.add_argument('--update-from-date')
pars.add_argument('--debug', action='store_true')
pars.add_argument('--debug-sql', action='store_true')
return pars
def get_option(argv):
pars = get_parser()
return pars.parse_args(argv)
class Conf(BaseConf):
def as_datetime(self):
pola = '%d-%m-%Y %H:%M:%S'
......@@ -240,46 +215,6 @@ class Conf(BaseConf):
return datetime.strptime(self.nilai, pola)
def is_live(pid):
try:
os.kill(pid, 0)
except OSError:
return
return True
def read_pid_file(filename):
try:
f = open(filename)
s = f.read()
f.close()
s = s.split()
s = s[0]
return int(s)
except IOError:
return
except ValueError:
return
except IndexError:
return
def write_pid_file(filename):
pid = os.getpid()
f = open(filename, 'w')
f.write(str(pid))
f.close()
return pid
def make_pid_file(filename):
pid = read_pid_file(filename)
if pid and is_live(pid):
print(f'PID saya {pid} masih aktif.')
return
return write_pid_file(filename)
def create_session(db_url, debug=False):
engine = create_engine(db_url, echo=debug)
factory = sessionmaker(bind=engine)
......@@ -291,29 +226,20 @@ class BaseApp:
report_orm = None # Override, please
va_product_code = '' # Override, please
def __init__(self, argv):
self.option = self.get_option(argv)
self.conf_all = ConfigParser()
self.conf_all.read(self.option.conf)
self.conf = self.conf_all['main']
self.pid = make_pid_file(self.conf['pid_file'])
if not self.pid:
return
self.prod_session = self.handler = self.odbc_conn = self.models = None
self.log = create_log(self.conf['log_file'])
def __init__(self, conf):
self.conf = conf
self.prod_session = self.models = None
factory = self.get_factory('report_db_url')
self.rpt_session = factory()
register(self.rpt_session)
if 'models' in self.conf:
self.models = get_module_object(self.conf['models'])
if 'service' in self.conf:
self.service = get_module_object(self.conf['service'])
try:
self.handler = self.service.Handler(self)
except AttributeError as e:
pass
module_conf = dict(self.conf)
if 'db_url' in self.conf:
if self.conf['db_url'] == 'odbc':
import pyodbc
odbc_profile = dict(self.conf_all.items('odbc'))
odbc_profile = self.get_prefix_config('odbc.')
self.odbc_conn = pyodbc.connect(**odbc_profile)
else:
factory = self.get_factory('db_url')
......@@ -323,9 +249,6 @@ class BaseApp:
self.h2h_session = factory()
else:
self.h2h_session = None
factory = self.get_factory('report_db_url')
self.rpt_session = factory()
register(self.rpt_session)
if 'va_db_url' in self.conf:
factory = self.get_factory('va_db_url')
self.va_session = factory()
......@@ -336,12 +259,17 @@ class BaseApp:
else:
self.va_session = None
def get_option(self, argv):
return get_option(argv)
def get_prefix_config(self, prefix):
d = dict()
for key in self.conf:
if key.find(prefix):
k = key[len(prefix):]
d[k] = self.conf[key]
return d
def get_factory(self, name):
db_url = self.conf[name]
engine = create_engine(db_url, echo=self.option.debug_sql)
engine = create_engine(db_url)
return sessionmaker(bind=engine)
def get_last_id(self, nama):
......@@ -375,6 +303,7 @@ class BaseApp:
q = self.get_payment_query()
no = self.offset
found = False
log = getLogger('update_from_date()')
for pay in q.offset(self.offset).limit(row_limit):
found = True
no += 1
......@@ -400,10 +329,10 @@ class BaseApp:
msg = f'INSERT {d}'
rpt = self.report_orm(**source)
msg = f'{self.get_prefix_log()} {msg}'
log_method = self.log.info
log_method = log.info
except InvalidSource as e:
msg = str(e)
log_method = self.log.warning
log_method = log.warning
rpt = None
e = self.get_estimate(no)
log_method(f'#{no}/{self.count} {msg}, estimate {e}')
......@@ -419,9 +348,6 @@ class BaseApp:
pass
def update_last(self):
if self.handler:
self.last.nilai = self.handler.get_last_time()
else:
self.last.nilai = self.get_last_time()
with transaction.manager:
self.rpt_session.add(self.last)
......@@ -429,45 +355,29 @@ class BaseApp:
def get_count(self): # Override, please
pass
def run_payment(self):
self.last_pay = None
def run(self):
self.last_pay = self.last = None
self.offset = 0
self.tgl_akhir = date.today()
if self.option.update_from_date:
self.last = None
try:
days_ago = int(self.option.update_from_date)
self.tgl_awal = date.today() + timedelta(days_ago)
except ValueError:
t = self.option.update_from_date.split(',')
self.tgl_awal = datetime.strptime(t[0], '%d-%m-%Y')
if t[1:]:
self.tgl_akhir = datetime.strptime(t[1], '%d-%m-%Y')
if 'tgl_awal' in self.conf:
self.tgl_awal = self.conf['tgl_awal']
else:
self.last = self.get_last_id(self.conf_name)
self.tgl_awal = self.last.as_datetime()
while True:
if self.handler:
self.count = self.handler.get_count()
if 'tgl_akhir' in self.conf:
self.tgl_akhir = self.conf['tgl_akhir']
else:
self.tgl_akhir = date.today()
while True:
self.count = self.get_count()
if not self.count:
return
self.start_time = time()
if self.handler:
found = self.handler.update_from_date()
else:
found = self.update_from_date()
if not found:
break
if self.last_pay and self.last:
self.update_last()
def run(self):
if not self.pid:
return
self.run_payment()
def get_va_channel(self, pay_date):
if not self.va_session:
return
......@@ -476,6 +386,7 @@ class BaseApp:
row = q.first()
if not row:
return
log = getLogger('get_va_channel()')
va_pay, va_inv = row
va_pay_date = datetime.strptime(
va_pay.transaction_date, '%Y-%m-%d %H:%M:%S')
......@@ -484,7 +395,7 @@ class BaseApp:
msg = f'Invoice ID {self.invoice_id} ada pembayaran melalui '\
f'VA/QRIS tapi tanggalnya beda yaitu {pay_date} vs '\
f'{va_pay_date}'
self.log.warning(msg)
log.warning(msg)
return
if va_inv.va_type == 'a':
return 'VA'
......@@ -493,18 +404,3 @@ class BaseApp:
raise Exception(
f'Invoice ID {invoice_id} va_type {inv.va_type} '
'belum dipahami, perbaiki script')
def init_db(argv=sys.argv[1:]):
conf_file = argv[0]
conf = ConfigParser()
conf.read(conf_file)
db_url = conf.get('main', 'report_db_url')
engine = create_engine(db_url, echo=True)
Base.metadata.create_all(engine)
BaseConf.metadata.create_all(engine)
factory = sessionmaker(bind=engine)
my_registry['db_session'] = db_session = factory()
register(db_session)
with transaction.manager:
append_csv(Conf, 'conf.csv', ['nama'])
payment_report/models.py
......@@ -118,11 +118,11 @@ class Bphtb(Base, Common):
# Bit 61
nop = Column(String(22), nullable=False)
# bphtb.bphtb_bank.wp_nama
wp_nama = Column(String(50), nullable=False)
wp_nama = Column(String(225), nullable=False)
# bphtb.bphtb_bank.wp_alamat
wp_alamat = Column(String(100), nullable=False)
wp_alamat = Column(String(225), nullable=False)
# bphtb.bphtb_sspd.op_alamat
op_alamat = Column(String(100), nullable=False)
op_alamat = Column(String(225), nullable=False)
# bphtb.bphtb_bank.npop
npop = Column(Float, nullable=False)
# bphtb.bphtb_bank.bumi_luas
......@@ -141,6 +141,7 @@ class Bphtb(Base, Common):
channel_nama = Column(String(32), nullable=False)
# bphtb.bphtb_reversal.tgl
tgl_batal = Column(DateTime(timezone=True))
status_pembayaran = Column(Integer)
class Pbb(Base, Common):
......
payment_report/scripts/pad.py → payment_report/pad/default.py
import sys
from datetime import (
date,
datetime,
)
from sqlalchemy import func
from sqlalchemy.exc import ProgrammingError
from opensipkd.string import FixLength
from opensipkd.waktu import dmyhms
from iso8583_web.models.meta import Base as BaseConf
from opensipkd.iso8583.bjb.pad.structure import PAYMENT_CODE
from opensipkd.iso8583.bjb.pad.doc import Doc as BjbDoc
from opensipkd.iso8583.bjb.pad.models import Log
from opensipkd.iso8583.multi.doc import Doc as MultiDoc
from ..models import (
Base,
Pad,
)
from .common import (
from ..models import Pad
from ..common import (
get_iso,
get_channel_info_by_iso,
get_channel_name_by_row,
BaseApp,
init_db as base_init_db,
one_day,
BANK_NAMES,
)
......@@ -41,10 +31,6 @@ class App(BaseApp):
def __init__(self, argv):
super().__init__(argv)
if not self.pid:
return
if self.handler:
return
Usaha = self.models.Usaha
Pajak = self.models.Pajak
Customer = self.models.Customer
......@@ -100,7 +86,7 @@ class App(BaseApp):
channel = BANK_NAMES[str(row.bank_id)]
return '0000', channel, None, None, row.bank_id
Doc = BankHandlers[row.bank_id]
iso = get_iso(row.iso_request, Doc, self.option.debug)
iso = get_iso(row.iso_request, Doc)
info = get_channel_info_by_iso(iso)
return info.get('bit_018', '0000'), info['channel'], iso.get_stan(), \
iso.get_ntb(), row.bank_id
......@@ -153,13 +139,3 @@ class App(BaseApp):
pokok=pay.jml_bayar-pay.denda, denda=pay.denda, bunga=pay.bunga,
jml_bayar=pay.jml_bayar, channel_id=channel_id,
channel_name=channel_name, bank_id=bank_id)
def main(argv=sys.argv[1:]):
app = App(argv)
if app.pid:
app.run()
def init_db(argv=sys.argv[1:]):
base_init_db(Base.metadata, BaseConf.metadata, argv)
payment_report/scripts/pbb.py → payment_report/pbb/default.py
......@@ -24,7 +24,6 @@ from opensipkd.waktu import (
dmyhms,
create_datetime,
)
from opensipkd.iso8583.bjb.scripts.common import get_module_object
from opensipkd.iso8583.bjb.pbb.default import Doc
from opensipkd.iso8583.bjb.pbb.structure import PAYMENT_CODE
from opensipkd.iso8583.bjb.pbb.models import Log
......@@ -33,7 +32,7 @@ from ..models import (
Pbb,
Base,
)
from .common import (
from ..common import (
get_iso,
get_channel_info_by_iso,
get_channel_name_by_row,
......@@ -103,8 +102,7 @@ class App(BaseApp):
def __init__(self, argv):
super().__init__(argv)
if not self.pid:
return
self.kd_tp_kasda = self.kd_tp_va = self.kd_tp_qris = None
self.set_other_db()
if self.prod_session:
d = Doc()
......@@ -113,12 +111,12 @@ class App(BaseApp):
self.prod_session.query(d.iso_payment_model).first()
self.iso_payment_orm = d.iso_payment_model
except DatabaseError:
self.prod_session.rollback()
self.iso_payment_orm = AlternativePayment
self.base_q_iso = self.prod_session.query(
self.iso_payment_orm, self.iso_inquiry_orm).filter(
self.iso_payment_orm.inquiry_id == self.iso_inquiry_orm.id)
if 'models' in self.conf:
self.models = get_module_object(self.conf['models'])
self.Psppt = self.models.PembayaranSppt
self.Sppt = self.models.Sppt
self.base_q_sppt = self.prod_session.query(self.Sppt)
......@@ -359,8 +357,3 @@ class App(BaseApp):
q = self.prod_session.query(self.Psppt)
q = self.get_filter_query(q)
return q.order_by(self.Psppt.tgl_rekam_byr_sppt)
def main(argv=sys.argv[1:]):
app = App(argv)
app.run()
payment_report/pbb/sukabumi_kota.py 0 → 100644
from ISO8583.ISO8583 import BitNotSet
from sismiop.services.sukabumi_kota import get_id
from sismiop.models.sukabumi_kota import (
Sppt,
PembayaranSppt as Psppt,
)
from opensipkd.waktu import dmyhms
from opensipkd.iso8583.bjb.pbb.default import Doc
from opensipkd.iso8583.bjb.pbb.default.models import Inquiry
from ..models import Pbb
from ..common import (
get_iso,
get_channel_info_by_iso,
get_channel_name_by_row,
get_channel_name_by_dict,
one_day,
InvalidSource,
)
from .default import (
App as BaseApp,
AlternativePayment as Payment,
to_date,
)
class App(BaseApp):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.iso_inquiry_orm = Inquiry
self.iso_payment_orm = Payment
self.base_q_iso = self.prod_session.query(
self.iso_payment_orm, self.iso_inquiry_orm).filter(
self.iso_payment_orm.inquiry_id == self.iso_inquiry_orm.id)
self.base_q_sppt = self.prod_session.query(Sppt)
def get_sppt(self, psppt):
q = self.base_q_sppt.filter_by(
KD_PROPINSI=psppt.KD_PROPINSI,
KD_DATI2=psppt.KD_DATI2,
KD_KECAMATAN=psppt.KD_KECAMATAN,
KD_KELURAHAN=psppt.KD_KELURAHAN,
KD_BLOK=psppt.KD_BLOK,
NO_URUT=psppt.NO_URUT,
KD_JNS_OP=psppt.KD_JNS_OP,
THN_PAJAK_SPPT=psppt.THN_PAJAK_SPPT)
return q.first()
def get_iso_row(self, psppt):
iso_inq = self.iso_inquiry_orm
awal = psppt.TGL_REKAM_BYR_SPPT.date()
akhir = awal + one_day
q = self.get_iso_query(psppt)
q = q.filter(iso_inq.tgl >= awal, iso_inq.tgl < akhir)
return q.first()
def get_iso_query(self, psppt):
iso_inq = self.iso_inquiry_orm
q = self.base_q_iso.filter_by(
propinsi=psppt.KD_PROPINSI,
kabupaten=psppt.KD_DATI2,
kecamatan=psppt.KD_KECAMATAN,
kelurahan=psppt.KD_KELURAHAN,
blok=psppt.KD_BLOK,
urut=psppt.NO_URUT,
jenis=psppt.KD_JNS_OP,
tahun=psppt.THN_PAJAK_SPPT,
ke=psppt.PEMBAYARAN_SPPT_KE)
return q.order_by(iso_inq.id.desc())
def warning_date(self, psppt):
q = self.get_iso_query(psppt)
row = q.first()
if row:
pay, inq = row
msg = f'Invoice ID {self.invoice_id} berbeda tanggal dengan '\
f'ISO8583 yaitu {psppt.TGL_REKAM_BYR_SPPT.date()} vs '\
f'{inq.tgl}'
self.log.warning(msg)
def get_iso_v1(self, psppt):
if self.h2h_session:
return
row = self.get_iso_row(psppt)
if not row:
self.warning_date(psppt)
return
row_pay, row_inq = row
iso = get_iso(row_pay.iso_request, Doc)
info = get_channel_info_by_iso(iso)
try:
user_id = iso.getBit(107)
except BitNotSet:
user_id = None
try:
channel_id = iso.get_channel().strip()
except BitNotSet:
channel_id = '0000'
return channel_id, info['channel'], iso.get_stan(), \
iso.get_ntb(), str(iso.get_bank_id()), user_id, row_inq.tgl
def get_source(self, psppt):
source = self.get_iso_v1(psppt)
if source:
channel_kode, channel_nama, stan, ntb, bank_id, user_id, \
tgl_inquiry = source
else:
stan = ntb = bank_id = user_id = tgl_inquiry = None
channel_kode = '0000'
channel_nama = 'MANUAL'
return channel_kode, channel_nama, stan, ntb, bank_id, user_id, \
tgl_inquiry
def create_data(self, psppt): # Override
self.invoice_id = get_id(psppt)
sppt = self.get_sppt(psppt)
if not sppt:
msg = f'Invoice ID {self.invoice_id} tidak ada di tabel sppt'
raise InvalidSource(msg)
channel_kode, channel_nama, stan, ntb, bank_id, user_id, \
tgl_inquiry = self.get_source(psppt)
bayar = psppt.JML_SPPT_YG_DIBAYAR
denda = psppt.DENDA_SPPT or 0
pokok = sppt.PBB_YG_HARUS_DIBAYAR_SPPT
tgl = to_date(psppt.TGL_PEMBAYARAN_SPPT)
if ntb:
ntb = ntb.strip()
return dict(
kd_propinsi=psppt.KD_PROPINSI, kd_dati2=psppt.KD_DATI2,
kd_kecamatan=psppt.KD_KECAMATAN, kd_kelurahan=psppt.KD_KELURAHAN,
kd_blok=psppt.KD_BLOK, no_urut=psppt.NO_URUT,
kd_jns_op=psppt.KD_JNS_OP, thn_pajak_sppt=psppt.THN_PAJAK_SPPT,
pembayaran_sppt_ke=psppt.PEMBAYARAN_SPPT_KE, stan=stan, ntb=ntb,
jml_sppt_yg_dibayar=bayar, denda_sppt=denda, discount=0,
tgl_pembayaran_sppt=tgl, tgl_inquiry=tgl_inquiry,
tgl_rekam_byr_sppt=psppt.TGL_REKAM_BYR_SPPT,
nm_wp_sppt=sppt.NM_WP_SPPT, channel_kode=channel_kode,
channel_nama=channel_nama, bank_id=bank_id, user_id=user_id,
pbb_yg_harus_dibayar_sppt=pokok)
def get_report(self, psppt): # Override
q = self.rpt_session.query(Pbb).filter_by(
kd_propinsi=psppt.KD_PROPINSI,
kd_dati2=psppt.KD_DATI2,
kd_kecamatan=psppt.KD_KECAMATAN,
kd_kelurahan=psppt.KD_KELURAHAN,
kd_blok=psppt.KD_BLOK,
no_urut=psppt.NO_URUT,
kd_jns_op=psppt.KD_JNS_OP,
thn_pajak_sppt=psppt.THN_PAJAK_SPPT,
pembayaran_sppt_ke=psppt.PEMBAYARAN_SPPT_KE)
return q.first()
def get_last_time(self): # Override
return dmyhms(self.last_pay.TGL_REKAM_BYR_SPPT)
def get_filter_query(self, q): # Override
return q.filter(
Psppt.TGL_REKAM_BYR_SPPT >= self.tgl_awal,
Psppt.TGL_REKAM_BYR_SPPT < self.tgl_akhir + one_day)
def get_payment_query(self): # Override
q = self.prod_session.query(Psppt)
q = self.get_filter_query(q)
return q.order_by(Psppt.TGL_REKAM_BYR_SPPT)
payment_report/scripts/init_db.py 0 → 100644
import sys
from configparser import ConfigParser
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from zope.sqlalchemy import register
import transaction
from opensipkd.views.models import Conf as BaseConf
from ..models import Base
from ..common import (
my_registry,
append_csv,
)
def main(argv=sys.argv[1:]):
conf_file = argv[0]
conf = ConfigParser()
conf.read(conf_file)
db_url = conf.get('main', 'report_db_url')
engine = create_engine(db_url, echo=True)
Base.metadata.create_all(engine)
BaseConf.metadata.create_all(engine)
factory = sessionmaker(bind=engine)
my_registry['db_session'] = db_session = factory()
register(db_session)
with transaction.manager:
append_csv(BaseConf, 'conf.csv', ['nama'])
payment_report/scripts/logger.py 0 → 100644
from configparser import RawConfigParser
import logging
import logging.config
def setup_logging(conf_file):
conf = RawConfigParser()
conf.read(conf_file)
d = {
'version': 1,
'formatters': {},
'handlers': {},
'loggers': {},
}
for section in conf.sections():
if section.find('formatter_') == 0:
name = section.split('formatter_')[1]
data = {'format': conf.get(section, 'format')}
d['formatters'][name] = data
elif section.find('handler_') == 0:
name = section.split('handler_')[1]
data = {'formatter': conf.get(section, 'formatter')}
data['class'] = 'logging.' + conf.get(section, 'class')
if conf.has_option(section, 'stream'):
data['stream'] = 'ext://' + conf.get(section, 'stream')
else:
data['filename'] = conf.get(section, 'filename')
d['handlers'][name] = data
elif section.find('logger_') == 0:
name = section.split('logger_')[1]
if name == 'root':
name = ''
data = {'level': conf.get(section, 'level')}
value = conf.get(section, 'handlers')
data['handlers'] = [x.strip() for x in value.split(',')]
d['loggers'][name] = data
logging.config.dictConfig(d)
payment_report/scripts/sync.py 0 → 100644
import sys
import os
from configparser import ConfigParser
from argparse import ArgumentParser
from datetime import (
datetime,
date,
timedelta,
)
from opensipkd.iso8583.bjb.scripts.common import get_module_object
from .logger import setup_logging
def is_live(pid):
try:
os.kill(pid, 0)
except OSError:
return
return True
def read_pid_file(filename):
try:
with open(filename) as f:
s = f.read()
s = s.split()
s = s[0]
return int(s)
except IOError:
return
except ValueError:
return
except IndexError:
return
def write_pid_file(filename):
pid = os.getpid()
with open(filename, 'w') as f:
f.write(str(pid))
return pid
def make_pid_file(filename):
pid = read_pid_file(filename)
if pid and is_live(pid):
print(f'PID saya {pid} masih aktif.')
return
return write_pid_file(filename)
def get_parser():
pars = ArgumentParser()
pars.add_argument('conf')
pars.add_argument('--update-from-date')
return pars
def get_option(argv):
pars = get_parser()
return pars.parse_args(argv)
def main(argv=sys.argv):
option = get_option(argv[1:])
conf = ConfigParser()
conf.read(option.conf)
cf = dict(conf.items('main'))
if not make_pid_file(cf['pid_file']):
return
setup_logging(option.conf)
if option.update_from_date:
try:
days_ago = int(option.update_from_date)
cf['tgl_awal'] = date.today() + timedelta(days_ago)
except ValueError:
t = option.update_from_date.split(',')
cf['tgl_awal'] = datetime.strptime(t[0], '%d-%m-%Y')
if t[1:]:
cf['tgl_akhir'] = datetime.strptime(t[1], '%d-%m-%Y')
module = get_module_object(conf.get('main', 'module'))
app = module.App(cf)
app.run()
payment_report/scripts/webr.py → payment_report/webr/default.py
import sys
from datetime import (
date,
datetime,
)
from sqlalchemy import func
from sqlalchemy.exc import ProgrammingError
import transaction
......@@ -11,7 +6,6 @@ from opensipkd.webr.models.default import (
Payment,
Invoice,
)
from iso8583_web.models.meta import Base as BaseConf
from opensipkd.iso8583.bjb.webr.structure import PAYMENT_CODE
from opensipkd.iso8583.bjb.webr.models import (
LogMixin,
......@@ -21,9 +15,8 @@ from ..models import (
Base,
Webr,
)
from .common import (
from ..common import (
BaseApp,
init_db as base_init_db,
BIT_18_NAMES,
get_channel_name_by_row,
one_day,
......@@ -41,8 +34,6 @@ class App(BaseApp):
def __init__(self, argv):
super().__init__(argv)
if not self.pid:
return
self.base_q_pay = self.prod_session.query(Payment)
try:
self.base_q_pay.first()
......@@ -144,13 +135,3 @@ class App(BaseApp):
orm = self.IsoLog
q = self.get_filter_query(q)
return q.order_by(orm.created)
def main(argv=sys.argv[1:]):
app = App(argv)
if app.pid:
app.run()
def init_db(argv=sys.argv[1:]):
base_init_db(Base.metadata, BaseConf.metadata, argv)
pbb-odbc.ini deleted 100644 → 0
[main]
service = payment_report.pbb.tangkab
db_url = odbc
report_db_url = postgresql://user:pass@localhost/payment_report
pid_file = /home/sugiana/tmp/pbb-report.pid
log_file = /home/sugiana/tmp/pbb-report.log
[odbc]
driver = FreeTDS
server = 192.168.168.31
database = payswitch
port = 5000
uid = FIXME
pwd = FIXME
pbb.ini
[main]
module = payment_report.pbb.default
models = sismiop.models.default
db_url = postgresql://user:pass@localhost/db
#h2h_db_url = postgresql://user:pass@localhost/db
......@@ -7,3 +8,23 @@ report_db_url = postgresql://user:pass@localhost/db
pid_file = /home/sugiana/tmp/pbb-report.pid
log_file = /home/sugiana/log/pbb-report.log
nip_pospbb = 888888888
[formatter_generic]
format = %(asctime)s %(levelname)s %(name)s %(message)s
[formatter_simple]
format = %(asctime)s %(levelname)s %(message)s
[handler_console]
class = StreamHandler
stream = sys.stdout
formatter = generic
[handler_file]
class = FileHandler
filename = /home/sugiana/log/pbb-report.log
formatter = simple
[logger_root]
handlers = console, file
level = INFO
setup.py
......@@ -34,11 +34,8 @@ setup(
zip_safe=False,
entry_points={
'console_scripts': [
'payment_report_init_db = payment_report.scripts.common:init_db',
'pbb_report = payment_report.scripts.pbb:main',
'bphtb_report = payment_report.scripts.bphtb:main',
'pad_report = payment_report.scripts.pad:main',
'webr_report = payment_report.scripts.webr:main',
'payment_report_init_db = payment_report.scripts.init_db:main',
'payment_report = payment_report.scripts.sync:main',
'pbb_log = payment_report.scripts.pbb_log:main',
'log2iso_init_db = payment_report.scripts.log2iso_init:main',
'log2iso = payment_report.scripts.log2iso:main',
......
sybase-requirements.txt deleted 100644 → 0
sqlalchemy-sybase < 2
webr.ini
[main]
module = payment_report.webr.default
db_url = postgresql://user:pass@localhost/db
#va_db_url = postgresql://user:pass@localhost/db
report_db_url = postgresql://user:pass@localhost/db
pid_file = /home/sugiana/tmp/webr-report.pid
log_file = /home/sugiana/log/webr-report.log
[formatter_generic]
format = %(asctime)s %(levelname)s %(name)s %(message)s
[formatter_simple]
format = %(asctime)s %(levelname)s %(message)s
[handler_console]
class = StreamHandler
stream = sys.stdout
formatter = generic
[handler_file]
class = FileHandler
filename = /home/sugiana/log/webr-report.log
formatter = simple
[logger_root]
handlers = console, file
level = INFO
Markdown is supported
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!